Skip to content

Inherited failures fail the Run without a Repair #103

Closed

Description

@JacobStephens2

Parent

#221

What to build

When CI goes red, a Run compares the failed checks on its head with the checks on the Base branch commit that head last merged in, by check name. A check that is Failed on both is an Inherited failure (see ADR 0008 and CONTEXT.md).

  • If every red check is an Inherited failure, no Repair starts:
    • if the Base branch has moved since the merge, the Run merges it in and watches CI again, counted as a Base move;
    • otherwise the Run is a Failed run with the cause CI red on <check>[, <check>…], which also fails on <base> at <short sha>; fix <base> first, and its pull request goes back to a draft.
  • If some red checks are Inherited failures and some aren't, the CI-fix Repair is given only the branch's own failures. Its prompt lists the Inherited failures separately as "also failing on <base>; don't fix". If only Inherited failures are left after it, the rule above applies.
  • A check that is pending, passed or missing on the Base branch commit is not an Inherited failure, and behaves as today. thirdshift never triggers or waits for the Base branch's CI, and never reads log text.
  • This applies wherever the Repair loop runs: plain Runs, Merge runs (no Self-merge past it), Ticket Runs against the Spec branch, and the Spec PR.

No Base fix in this Ticket.

Acceptance criteria

  • A Run whose only failing checks also fail on the Base branch commit ends as a Failed run with no CI-fix Repair session, its cause names each check, the Base branch and the short sha, and the pull request is a draft.
  • A Merge run in the same state does not Self-merge.
  • A Run with a check that fails only on the branch behaves as it does today; the existing CI tests pass unchanged.
  • With mixed failures, the CI-fix Repair prompt lists only the branch's own failures as ones to fix, and names the Inherited failures as not to fix.
  • A check that is pending or missing on the Base branch commit is handed to a CI-fix Repair as today.
  • When the Base branch moved since the merge, the Run merges it and watches CI again instead of failing.
  • A Spec run's Ticket meeting an Inherited failure on the Spec branch shows the cause in the Tickets checklist.
  • Tested through the binary with the fake gh and claude, setting checks on both commits with gh fake checks.

Blocked by

None (can start immediately).

Original triage brief

The policy question this issue asked was decided in a grilling session: option 1, report only. See the comments, ADR 0008 and #221.

Activity

  1. JacobStephens2 commented on Sep 30, 2026

    @JacobStephens2
    OwnerAuthor

    This was generated by AI during triage.

    Second case: JacobStephens2/chart35#71 (Spec run for #61, 2026-09-30)

    Verify failed on 37846c2 (run 36782270641) on one test, server/src/routes/entitlements.http.test.ts "publishes access ending instead of renewal for a canceling active subscription". Its fixture hardcoded current_period_end 2026-09-24, which had since passed. main's latest run (36736140334) failed on the same assertion at the same line, 194. The fix went to main separately in JacobStephens2/chart35#78.

    thirdshift started Repair 1 of 5. The Repair found the same failure on main using gh run list --branch main and gh run view --log-failed, declined to change code, and added a "CI notes" section to the PR body. The Run then ended with CI red on 37846c2 and the Repair found nothing to fix on the branch. #102 is working as intended here: one Repair, not five.

    Proposal for the open question "does thirdshift check on its own?"

    Check the Base branch before starting the CI-fix Repair, not inside it. When a check fails, thirdshift looks at the latest completed run of the same workflow on origin/<base>. If the same check failed there too, it skips the Repair and ends with a cause like CI red on Verify, which also fails on main (run 36736140334); fix main first.

    • Saves a Repair session. This one cost $0.35 and 51s, only to reach a conclusion thirdshift could have gotten from two gh calls.
    • Matches by conclusion per check name, not by log text. That keeps CONTEXT.md's rule against reading GitHub's error text. The trade-off: if the branch has a different failure in the same check, it hides behind the Base branch's red. Options 1 and 2 above can take that risk because neither merges.
    • Leaves the policy question open. The ADR still has to pick options 1, 2, or 3. This proposal is only about where detection happens.
  2. JacobStephens2 commented on Oct 1, 2026

    @JacobStephens2
    OwnerAuthor

    Decided in a grilling session. Recorded in ADR 0008 and in CONTEXT.md (Inherited failure, Base fix), PR #220, with the Spec in #221.

    • Policy: report only. A check that also fails on the Base branch commit the head merged in makes the Run a Failed run with no Repair, and its cause says fix <base> first. Nothing merges past red CI.
    • Detection: thirdshift does it before any Repair, by check name and conclusion, never from log text. A check with no finished result on the Base branch isn't an Inherited failure.
    • Opt-in Base fix: a Merge run into the Base branch that fixes the check. It comes in later Tickets of Inherited failures: fail fast on a red Base branch, or fix it with a Base fix #221.

    This issue is now the first Ticket of #221: detection and reporting only.

  3. added
    ready-for-agentFully specified, ready for an AFK agent
    and removed on Oct 1, 2026
  4. changed the title [-]Decide how a Run treats a CI check that is also red on the Base branch[/-] [+]Inherited failures fail the Run without a Repair[/+] on Oct 1, 2026
  5. JacobStephens2 commented on Oct 1, 2026

    @JacobStephens2
    OwnerAuthor

    Closed by #225, merged into issue-221 by a thirdshift Merge run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions