Repository navigation
Stop harness waits failing on a slow machine - #227
Merged
Merged
Conversation
A test that waits for a Run to start the agent, or to show a prompt on the terminal, now fails as soon as the Run exits without getting there, with the Run's stderr, and otherwise waits up to two minutes instead of ten seconds. The interrupt tests prove the session was stopped by a marker its script would write after its sleep, not by how long the test took, and the Spec run sessions that wait for each other allow two minutes too. Closes #200 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The harness tests assert the wait failed because the Run exited, not how long it took. The Spec run scripts derive their polling from WAIT_BOUND, and the harness helpers document when they panic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The harness's two waits on a Run, for the fake agent to start (
Scenario::run_and_signal) and for a prompt on the terminal (Scenario::run_on_terminal), no longer give up after a fixed 10s.Three sibling timings in the tests went the same way:
tests/failed_run.rs,tests/spec_run.rs: the interrupt tests asserted the whole Run took under 20s, which now includes however long the agent took to start. They assert instead that the session's script never got past itssleep 30.tests/spec_run.rs: sessions that wait in bash for another Ticket's session to start waited 10s; they now waitWAIT_BOUND.Seams under test:
Scenario::run_and_signalandScenario::run_on_terminal, in the newtests/harness.rs.Evidence
ctrl_c_after_the_merge_finishes_the_post_merge_steps_and_exits_as_mergedpassed three times out of three. This stood in for "cargo testin two other checkouts", which was not run.cargo testpassed, 173.81s wall including test compilation, with load average about 4 from other sessions, so there is no clean idle before/after comparison. The newtests/harness.rsadds 0.17s; nothing else adds time. The full suite ran before the review fixes; after them,harness,failed_runandspec_run(every file the fixes touched) were rerun and passed, andcargo clippy --all-targetsis clean.Unaddressed findings
Standards
tests/support/mod.rsshare a shape. Skipped: on exit one collects piped output and the other drains the terminal before searching it, so a shared helper would take a callback per step and read worse than two short loops.tests/failed_run.rsandtests/spec_run.rs. Skipped: the duplication predates this change, and one sets the script for every session while the other sets it for one Ticket.SETUP_OFFERintests/harness.rsrepeatsOFFERintests/setup_offer.rs. Skipped: each test file is its own crate, and the harness module is not the place for one prompt's wording.let exited = loop { let exited = ... }shadows itself. Skipped: both hold the same fact, and the inner one is the value the loop breaks with.Spec
reader.join()andwait_with_output()have no deadline if a descendant still holds the terminal or the pipe. Skipped: the harness already made both calls unbounded at the end of each helper, and a Run that exits before starting an agent leaves no descendants.run_and_signal.cargo testrunning in two other checkouts" and "takes no longer on an idle machine" not verified as written. See Evidence for what was run instead.Merge Danger
Door: two-way
Tests only; no change to the binary.
Blast Radius: tests
A Run that truly hangs before reaching the awaited point now holds its test for 120s, not 10s, before failing.
Closes #200
🤖 Generated with Claude Code