Repository navigation
A Ticket fails as CI red when its Repair already re-ran the failed checks and they passed #508
Description
What happened
In the Spec run for #486 last night, Ticket #489 failed with "CI red on 7a2e53a and the Repair found nothing to fix on the branch", although CI on 7a2e53a was green by then.
From the command log (~/.thirdshift/logs/JacobStephens2/thirdshift/commands/pickup/486-20261007T013901-0400.log, times EDT):
- CI run 37581771582, attempt 1, failed on 7a2e53a. The failure was a flaky test (
two_tickets_meeting_the_same_inherited_failure_share_one_base_fix, Flaky spec_run scenario tests: concurrent Tickets race on git fetch and branch deletion #509). - 02:41:56: the Repair session itself ran
gh run rerun 37581771582 --failed. Attempt 2 passed, completing at 02:44:54. - 02:49:24: the Repair session ended, having changed nothing.
- 02:49:25: thirdshift: "re-running the failed checks on 7a2e53a: test"
- 02:49:26: "the failed checks on 7a2e53a were not re-run: gh run rerun 37581771582 --failed --repo JacobStephens2/thirdshift failed: run 37581771582 cannot be rerun; its workflow file may be broken"
- 02:49:29: "Reviewers list the files they read #489 failed", and thirdshift keeps each reviewer's report beside the Session log #490 was blocked by it.
PR #498 is now green and mergeable, but the Spec run had already stopped. Because issue-486 and PR #496 exist, #486 is no longer a Ready issue, so no Pickup run resumes it.
Expected
When gh run rerun --failed is refused, thirdshift should read the checks on the head commit again before calling the Ticket failed. Here the run had already been re-run, its new attempt passed, and the Ticket should have gone on to merge.
Where
src/ci.rs, rerun_through: if rerun_failed_jobs fails, it returns Ok(None) without looking at the checks' current state. The failed checks it acts on were read before the Repair session ran, so they predate any re-run the session made.
Notes
- A Repair session re-running CI is reasonable: its prompt lets it, and here it was the right fix for a flaky test.
- Either thirdshift picks up re-runs it didn't start, or the Repair prompt tells the session to leave CI re-runs to thirdshift. The first is more robust.
Activity
JacobStephens2 commentedon Oct 7, 2026
Agent Brief
Category: bug
Summary: A refused Check re-run must refresh CI on the unchanged head before a Run is failed as a Declined CI fix.
Current behavior:
The Repair loop retains the failed checks it read before starting a CI-fix Repair. When the Repair leaves the head unchanged, the Check re-run uses that snapshot. If GitHub refuses
gh run rerun --failed,rerun_throughreturnsNoneimmediately, and the Repair loop fails the Run as CI red. A Repair can already have re-run those checks and watched them pass, so the retained failures need not describe current CI.Verification:
7a2e53afdf7e6933df8bf790b211c40694041466, completed by 02:44:54 EDT. Attempt 1 failed on that same head.f03ac2b7358f2380940bd45c03eb65b5c67d091dusing the existingScenarioharness, real git, and fake GitHub/Harness executables: the implement session creates a failing Actions check whose re-run passes; session 2 runsgh run rerun 900 --failed --repo acme/widgetsand commits nothing. The fake GitHub then refuses thirdshift's second request because no failed job remains. Assertions confirmed the check was successful, the Run nevertheless exited unsuccessfully with the Declined CI fix cause, two re-run requests were made, and only one Repair ran.cargo test --test triage_508_repro -- --nocapturepassed its assertions of the broken behavior. The temporary reproduction test was removed after verification; no fix was implemented during triage.Desired behavior:
When a Check re-run request fails, determine the outcome from freshly read checks on the same head instead of immediately treating the earlier failure snapshot as current. If those checks have passed, continue normal Delivery and, for a Merge run, Self-merge. If they are pending, watch them finish using the existing interruption behavior. If the branch's own checks are still red, preserve the Declined CI fix outcome without spending another Repair or issuing another re-run for that head. Preserve the existing treatment of Inherited failures and the protection against reading an earlier attempt as the result of a successful re-run request.
The refusal remains visible in progress output, but its error text is not evidence of CI's current state. A failed checks read or an empty listing after previously known failures must not become an unverified successful outcome. When several workflow runs are involved, assess the whole head's CI, including any re-run already started before a later request was refused.
Key interfaces:
rerun/rerun_throughand theirResult<Option<Ci>>contract: a request refusal alone must not mean there is no current CI outcome to observe.Outsideboundary,checks_on, and watch behavior: reuse deterministic head-commit checks and polling, including current pending checks and Inherited failure classification.InMemoryandScenariotest boundaries can simulate a refused request with checks already green or pending.Acceptance criteria:
Out of scope: