Repository navigation
Testing with remote mode is not stable #118
Description
When testing locally with Ruby 3.1.0-dev, I sometimes get errors like this:
Error: test_break_with_namespaced_class_method_stops_at_correct_place(DEBUGGER__::BreakAtMethodsTest): Errno::EMFILE: Too many open files - fork failed
/Users/st0012/projects/debug/test/support/utils.rb:77:in `spawn'
/Users/st0012/projects/debug/test/support/utils.rb:77:in `block in new_child_process'
Or timeout error:
Failure: test_backtrace_prints_the_return_value(DEBUGGER__::BasicBacktraceTest):
TIMEOUT ERROR (10 sec)
[DEBUG SESSION LOG]
> [1, 10] in /var/folders/yg/hnbymwxd5pn7v94_clc59y6r0000gn/T/debugger20210701-59574-vo4b86.rb
> => 1| class Foo
> 2| def first_call
> 3| second_call(20)
> 4| end
> 5|
> 6| def second_call(num)
> 7| third_call_with_block do |ten|
> 8| num + ten
> 9| end
> 10| end
> =>#0 <main> at /var/folders/yg/hnbymwxd5pn7v94_clc59y6r0000gn/T/debugger20210701-59574-vo4b86.rb:1
> b 4
>
> (rdb) b 4
.
<false> is not true.
The master build also fails for timeout error. I can't rerun it but it looks like a random failure too.
Activity
st0012 commentedon Jul 1, 2021
@ono-max can you take a look? thanks
ono-max commentedon Jul 1, 2021
I think timeout error is not issue. I saw kind of this error before merging remote test, but I've never seen
Errno::EMFILE:. I'll check it.Since we started testing on remote, the number of tests increased by 3 times.That's why probability of random failure also increased.
ono-max commentedon Jul 2, 2021
Hmm... I can't reproduce the
Errno::EMFILE:. When I look into it, it may be able to be solved this problem by configuring your local environment.st0012 commentedon Jul 2, 2021
This issue happens a lot on Ruby
3.1.0-dev, have you tested against it?Even with remote tests, we only have < 500 assertions. It's not acceptable to just say "oh it fails sometimes".
Random failures make our tests unreliable and increase the chance to let real issues slip in. It also increases the maintainers' work because now the CI needs to reran very often.
st0012 commentedon Jul 2, 2021
In the recent 8 builds, 4 of them failed, including
master's. The 50% chance of failure is too high to accept.ono-max commentedon Jul 2, 2021
Sorry, I missed the version. I'll check it again.
fix part of #118
fix part of #118
ono-max commentedon Jul 5, 2021
I fixed
Errno::EMFILE: Too many open files - fork failed, so I'll continue to investigate how to fixTimeoutError.st0012 commentedon Jul 5, 2021
@ono-max that sounds great, thank you 👍
ono-max commentedon Jul 6, 2021
@st0012
I think we can close this issue because TimeoutError was fixed in #131.
Do you agree?
st0012 commentedon Jul 6, 2021
sure, thanks for al the fixes 😄