Repository navigation
Make the test suite faster - #210
Merged
Merged
Conversation
Line tables for the crate and no debuginfo for dependencies: the debug binary drops from 108MB to 27MB and target/ from 1.5GB to 838MB, and backtraces keep file and line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use line-table debuginfo for dev builds
axoupdater needs axoasset's remote feature, which always pulls in image and turns on reqwest's default-tls (rustls with aws-lc). No flag drops either, and asking reqwest for ring would build ring alongside aws-lc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The two --email looping tests in run_notification ran nine full Runs to cover flag position, bare flag against an address, the Issue URL never taken as an address, and merge before and after. args now has unit tests for every one of those cases, and each end-to-end test is cut to one Run showing the parsed arguments reach the Run notification. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Say why aws-lc and axoupdater's image stack stay
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unit tests for argument parsing; one Run per --email end-to-end test
Each looping test in spec_run, run_notification and release keeps one case end to end and moves the rest to unit tests where the cases differ: the interrupt's signals, the Spec run's Unready labels, the Resend key lookup, and the release scripts' version, summary and bump-PR steps, which the tests now call by sourcing the scripts. The Repair and upstream-move budget tests keep their real rounds, say so, and gain unit tests of the counting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Poll CI every 100ms in the test harness
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collapse the looping end-to-end scenarios to one Run per behavior
The warm suite missed both of #174's targets (249s wall, 650s user + sys against a 701s baseline), and the Python fakes' own CPU was about 352s of that: 4,940 gh calls and 381 claude calls. The fakes are now one std-only Rust executable, built from tests/fakes.rs with rustc the first time a test needs it and cached in CARGO_TARGET_TMPDIR by a hash of its sources. It acts as gh or claude by the name it is called by, under the same contract: the same state and record files, argv handling and output. tests/fakes.rs is also a Cargo test target, so clippy and rustfmt cover the fakes and the JSON layout is checked against Python's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pull the repeated pieces of the fakes into one place each: the --jq refusal, the "not open" check, the branch tip on origin, record appends and the lock file. Move the JSON tests next to the JSON module. Build the fakes without -D warnings, so a lint a newer rustc adds can't fail the suite: clippy covers them through the fakes test target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the Python fake gh and claude with a Rust helper
…led send after a failed Run The fakes now share one git runner and one reader for the FAKE_* paths, and `gh fake …` is handled by `fake_command`. A failed Run whose notification also fails to send is tested end to end again, as the collapsed loop covered it before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JacobStephens2
marked this pull request as ready for review
September 30, 2026 23:02
main's release script now waits for the release workflow run (#215), which taught the Python fake gh `run list --commit`, `run view`, `release view`, run attempts and $FAKE_MERGE_HEAD/$FAKE_MERGE_SHA. issue-174 replaced that fake with tests/fakes/gh.rs, so those changes are ported there and the Python fake stays deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This makes the test suite cheaper to build and run, so an agent session can run the full
cargo testin one foreground Bash call.The review found a few things, which this PR also fixes:
gitrunner and one reader for theirFAKE_*paths.gh fake …is now handled byfake_command.Evidence
Machine: 4 cores. None of these runs was on an idle machine.
cargo test, wallcargo test, user + syscargo test --no-run, wallI ran the warm suite again at this PR's head, after the review fixes: 127s wall, with 75s user + 156s sys = 231s CPU. All 25 test binaries pass.
The #207 runs also passed with a
python3stub first onPATHthat fails if called, so nothing calls Python any more. The per-ticket measurements are in the comments on #174.Merge Danger
Door: two-way
The only production-side changes are:
environment_keysplit out offrom_environmentinsrc/resend_key.rs, with the same behaviorscripts/release*.shnow runmainonly when executed, not when sourced, andis_plain_versionis extractedBlast Radius: tests
The shipped binary is unchanged: release builds keep their profile, and the TLS stack and axoupdater are untouched. The integration tests now need
rustconPATH(or$RUSTC) rather thanpython3. Dependency frames in dev backtraces lose file and line.Follow-up
remotefeature pins reqwest'sdefault-tls(aws-lc) andimage. Removing them needs either an upstream axoasset change or a replacement updater. Both are out of scope here. The details are in the Make the test suite faster #174 comment on Stop building aws-lc-sys, and axoupdater's image stack if a flag allows #203 and inCargo.toml.Unaddressed findings
Standards
Jsonreached by string keys. Kept on purpose: it keeps the Python fake's JSON contract, which the harness writes and reads.static LOCKintests/fakes/gh.rs, released byrun_hook. Kept: it mirrors how the Python fake let go of the lock while a hook ran, and the doc comment explains why.environment_keyexists only as a test seam. Kept: it is small and well named.Spec
aws-lc-sysis no longer built at all": not done. It can't be done with a feature flag (see Follow-up). The spec allows skipping a lever once the targets are met.src/run.rs.email.alwayswith no key, or with a key only in the Credentials, has no end-to-end test. Not re-added:main.rsturnsemail.alwaysinto the sameNotificationAskas--email, andconfig.rs's unit tests cover that. From there the key lookup is the same path, which is tested end to end.src/interrupt.rsshows SIGINT sets the interrupt flag. The Spec run then always sends SIGTERM to the Ticket's Run, and that path is tested end to end.Closes #174
Tickets
🤖 Generated with Claude Code