Repository navigation
Flaky spec_run scenario tests: concurrent Tickets race on git fetch and branch deletion #509
Description
What happened
Two tests/spec_run.rs scenario tests fail on Linux CI now and then with the Spec run exiting 1 instead of 0:
two_tickets_meeting_the_same_inherited_failure_share_one_base_fix(tests/spec_run.rs:855,left: Some(1),right: Some(0)). It failed CI for Reviewers list the files they read #489 (run 37581771582, attempt 1), which led to the Ticket failure in A Ticket fails as CI red when its Repair already re-ran the failed checks and they passed #508.without_a_limit_at_most_three_tickets_run_at_once(tests/spec_run.rs:1415,left: Some(1),right: Some(0)). It failed CI on the first 0.10.0 release PR (Release 0.10.0 #455).
Cause (for the first, from its stderr)
Tickets #21 and #22 run concurrently. #22 lands and its branch is deleted on origin ("#22: deleting issue-22 on origin"). At about the same time, #21 runs git fetch origin issue-20 and fails:
thirdshift: #21: git fetch origin issue-20 failed: fatal: bad object refs/heads/issue-22
thirdshift: #21: error: /tmp/.tmpSv00uD/origin.git did not send all necessary objects
thirdshift: #21 failed
The fetch reads origin's refs while refs/heads/issue-22 is being deleted, so origin advertises a ref that then can't be resolved. The second test's stderr hasn't been captured yet. It also runs Tickets concurrently, so the same race is the likely cause, but that needs confirming.
To decide
- Whether this can happen against GitHub, where ref deletion is atomic from a client's view, or only with the local bare
origin.gitthe scenarios use. - Fix options: make the fake origin's branch deletion atomic for concurrent fetches; retry a fetch once on "bad object" / "did not send all necessary objects"; or fetch only the needed refspec in a way that doesn't touch other refs.
Related
- Write unit-test executables from a child process #483 fixes the separate "Text file busy" unit-test flake. It also lists two other known flakes: the
harness::settingsinterrupt-timing tests, and the session test that runs a locally installedopencode.
Activity
JacobStephens2 commentedon Oct 7, 2026
Agent Brief
Category: bug
Summary: Prevent concurrent Ticket fetches and branch cleanup from intermittently failing Spec run scenarios.
Current behavior:
Two scenarios that expect a successful Spec run sometimes exit 1 when Ticket #21 fetches the Spec branch while Ticket #22 finishes. Both original Linux CI logs report
fatal: bad object refs/heads/issue-22, followed byorigin.git did not send all necessary objects:two_tickets_meeting_the_same_inherited_failure_share_one_base_fix: run 37581771582, attempt 1.without_a_limit_at_most_three_tickets_run_at_once: run 37533082167.Both used Git 2.55.0. The second scenario's matching stderr has now been verified; its relationship to the first is stronger than the original issue's hypothesis.
Verification during triage: Both named scenarios passed individually, once each, on the current checkout using Git 2.43.0. The intermittent failure was not reproduced locally; the original CI logs above confirm both failures. Passing these single runs does not establish that the race is fixed.
Scenarios redirect the GitHub origin URL to a local bare repository and Ticket Worktrees share the Launch directory's Git repository.
Gitretries existing lock-file and stale-ref-update conflicts, but not these errors. Worktree acquisition locks the Launch directory; subsequent fetches and Self-merge cleanup can overlap. The logs establish the failure and its overlap with cleanup, but do not establish whether the failing ref belongs to the origin or the shared local repository, or whether GitHub transport is affected.Desired behavior:
Concurrent Tickets can fetch the Spec branch and finish Self-merge cleanup without unrelated branch removal failing their Runs. Establish which Git operation and repository causes the race, then fix the narrowest supported cause. If the cause is specific to the scenario origin or transport, correct the fixture. If a production change is justified, document the evidence and preserve genuine failure reporting.
Key interfaces:
Git::fetchand its command execution/retry behavior.Scenarioorigin setup and fake GitHub merge behavior.Acceptance criteria:
Out of scope: