Repository navigation
Attribute unrecovered failures to the stage before trailing cleanup - #2140
Conversation
A Worktree dropped while a failure unwinds enters the cleanup stage before CommandRecord::finish runs, so failures without a recorded Recovery stage, and Where runs failed's fallback for older or died records, named cleanup instead of the stage that failed. Both now pass over trailing cleanup unless it is the only stage. Closes #2107
… name who enters cleanup in the guide
|
Delivery completed on agent-authored or reviewed head d6ec591. |
|
Review run. No fixes to push. Spec axis used the linked closing issues. Reviewed PR #2140 at Standards0 findings. No documented-standard violations or actionable baseline smells. The shared fallback keeps stage attribution in one place. Standards report and final files-read list Spec0 findings. The implementation matches linked closing issue #2107: trailing cleanup is skipped, cleanup-only records retain cleanup, and explicit Recovery stages retain precedence. Spec report and final files-read list No source fixes were needed. The working tree is clean; no commit was created. Unaddressed findings
No findings were declined or deferred; no triage issues were needed. Review coverageBoth reviewers directly inspected all eight changed files, including generated Each axis’s final files-read list consists of this shared list plus its additions below: Standards additionally read: Spec additionally read ValidationAll commands ran in this Review worktree.
Validation log · summary.json, containing Standards: 0 findings, worst issue: none. Spec: 0 findings, worst issue: none. SecurityNo unaddressed introduced findings. CI red at d6ec591
Inherited failures on main at 943166d: Repair 1No branch-owned fix was needed. The working tree is clean, and no commit was created. PR #2140 remains open, ready, and unchanged at The failure comes from an existing fixture-write race: rewriting the remote record briefly removes its row, then its return triggers “2 need you” before the third row arrives. Holding that empty interval for 350 ms reproduced the exact CI assertion. The test and involved runtime code are unchanged from Validation against that head:
The selector used CI passed or absent at d6ec591 |
Summary
When a failure unwinds out of
run,spec_runorfix, dropping the Worktree adds acleanupstage to the Command record beforeCommandRecord::finishruns. Two places use the record's stages to say where a Run stopped:finish, when there is no Recovery or its stage records nothing (Stage::BaseFix), and Where runs failed, for older or died records. Both took the last stage, so they reportedcleanupinstead of the stage that actually failed.Both now use one rule,
CommandRecord::stopped_stage, which replaceslast_stage:If
cleanupis the only stage, it is still reported.docs/guide.md(regenerated intosite/docs/index.html),site/architecture/index.htmland thewhere_runs_failedOpenAPI description now describe the new fallback.Evidence
Execution evidence
The host has no C linker, so Rust tests ran in the local
node:22Debian image, using the pinned 1.99.1 toolchain from the host.Cause confirmed.
impl Drop for Worktree(src/worktree.rs:544) callslogs::stage(Stage::Cleanup)beforeStarted::finishreachesCommandRecord::finish. The two seams below fail only when a trailingcleanupis present.cargo test --bin thirdshift a_command_records_its_version_at_start_and_where_a_failure_stopped_at_its_end(stagesimplement, CI watch, cleanup, recoveryNone)Stage::BaseFix, checks that an explicitSelfMergeRecovery still wins, and checks that a cleanup-only record still reportscleanup.cargo test --test metrics where_runs_failed_groups_recorded_endings(an unrecorded failed record with stagespull request, Self-merge, cleanup, and a died record with onlycleanup)Wider checks (
scripts/affected-tests mainselects the full Rust suite, the site checks and the Console browser tests; it was run by hand because neithercargo-nextestnor a linker is installed on the host):cargo test --locked --no-fail-fast: 4453 passed, 2 failed. Both failures also happen at the merge base built in its own target dir, so they come from the container, not this change:muse_sessions::muse_runs_unattended_…fails because its no-warning:check seescould not read process …'s environment: Permission denied.trial_scope::trial_sessions_keep_…fails becausesystemctlis missing.npm run docs:check --prefix site: OK.npm test --prefix site: 68/68 passed.npm test --prefix console(Playwright, run as root in a container): 198/205 passed. The same 7 host-name tests (32, 74, 124) fail on the merge-base binary as well, because the container hostname is random.cargo fmt --check: OK.Visual evidence
Only wording changes. Each capture is the changed paragraph at a 1440×900 viewport, served over HTTP.
site/architecture/(1440×900):site/docs/(user manual, 1440×900):Review
Standards: 3 findings. Spec: 3 findings. Fixed:
Endedconstruction in the test is now one shared closure.cleanupwhen its Worktree is dropped.The review did not read
site/docs/index.html. Both reviewers excluded it as generated output: its source isdocs/guide.md, its generator issite/generate-docs.mjs, andnpm run docs:check --prefix siteexits 0.Unaddressed findings
Standards
endingtest closure now takes four positional arguments. Declined. The stage list is each case's scenario input: a trailing cleanup is the condition under test. The reviewer also judged it "acceptable" in a test.Spec
last_stagewas renamed tostopped_stage, though the issue asked for no rename. No change.last_stagehad exactly two callers onmain, the two fallbacks this issue targets (src/logs/command_record.rs:481,src/logs/metrics/failures.rs:115). Its old meaning, "the stage entered last", is no longer what either caller wants.stopped_at: "cleanup"and afailure_categorykeep it. No change. The issue's Desired behavior limits the fallback to records "withoutstopped_at" and says "explicitrecovery.stage().recorded()values continue to take precedence unchanged".src/logs/metrics/failures.rs:113reads a savedstopped_atas recorded. So the existing v0.30.0cleanuprows stay, and new records are attributed correctly from this version on.src/logs/metrics/failures.rs:114-121→stopped_stage(), and onlyEndingdiffers. A failed cleanup-only record is covered throughfinishinsrc/logs/recording/tests.rs, and a died one by metrics fixture 16.CI notes
console-test:Working identifies live Review and Fix targets locally and across the Fleetfailed with390px remote review-explicit: visible PR target(run 43285536060). This is unrelated to the branch: it changes no Console code and doesn't touchconsole/tests/working-targets.test.cjs. The same assertion for the remotereview-explicitrow also fails on unrelated branches:issue-2052(run 43283026428, 390px) andissue-2018(run 43054228339, 1440px).Security review outcome
Merge Danger
Door: two-way
Nothing already saved is rewritten. Only newly finished records, and reads of unrecorded ones, change.
Blast Radius: metrics
Only where failed Commands are attributed changes:
stopped_atin new Command records and the Where runs failed fallback rows (local, Fleet and Console). Records that recorded nothing and ended with a trailingcleanupmove from thecleanuprow to the stage before it.Closes #2107
Built with claude 2.1.322 · claude-opus-5-5 · high