Repository navigation
test: stop CI test runs from timing out - #346
Merged
Merged
Conversation
The bugc example tests never let the worker's event loop turn, so on a slow runner vitest timed out waiting on its own progress reports. Yield after each test, and validate each distinct instruction context once instead of re-validating every copy.
gnidan
added
the
changelog: skip
Contributor
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
run-testshas been failing now and then withError: [vitest-worker]: Timeout calling "onTaskUpdate"even though every test passes. Both causes are in the bugc example tests (packages/bugc/test/examples/examples.test.ts).The timeout. The compiler and the hyperjump validator do all of their work synchronously, and vitest moves from one test to the next with promises only. So that whole file ran from start to end without the worker's event loop ever turning. The worker sends progress reports to the main process, but it cannot read the replies until the loop turns, and birpc gives up after 60 s. On CI the file takes about 64 s, so it was just over that line. Locally I measured the longest gap between event loop turns in that file at 15.0 s, which is the whole file. A file of seven 9 s busy tests reproduces the same error locally, and an
afterEachthat waits onsetImmediatemakes it go away. The examples file now has thatafterEach, which brings the longest gap down to 0.66 s (the slowest single test).The slow validation. The "Program validity" tests from #328 validated each whole program. A program lists all the variables in scope at every instruction, so
intermediate/arrays.bugcomes out at about 1.5 MB, and hyperjump took 1.5 s per program locally and 7 to 9 s on CI, against the 10 s limit. The schema was not being recompiled; a second validation of the same program takes just as long. But the contexts repeat: that program has 829 instructions and only 130 distinct contexts, and they repeat across optimization levels too. Each instruction'scontextis a plain$refto ethdebug/format/program/context, so the test now validates the program with the instruction contexts left out, and validates each distinct context once against the context schema (cached across tests). Errors are still reported at#/instructions/<n>/context/.... I checked that a bad context and a bad instruction field still fail the test.Measurements,
yarn test --maxWorkers=2locally:On CI, examples.test.ts went from 63.8 s to 17–25 s, and the slowest arrays.bug validity test from 9.1 s to 2.2–3.2 s (three runs of both Node jobs on this branch, all green).