Skip to content

Spec review before the Spec PR is ready - #164

Merged
JacobStephens2 merged 3 commits into
mainfrom
issue-142
Sep 30, 2026
Merged

JacobStephens2 merged 3 commits into
mainfrom
issue-142

Conversation

@JacobStephens2

Copy link
Copy Markdown
Owner

Summary

Once the last Ticket lands, a Spec run now runs a Spec review on the Spec branch before it marks the Spec PR ready.

 spec_run::run
   land_tickets                      # each Ticket a Merge run into the Spec branch
-  open_spec_pr                      # created ready for review
+  open_spec_pr                      # created as a draft (an open one is reused)
+  review
+    worktree.fast_forward_to_origin # take in the Tickets landed on origin
+    Sessions::run("spec-review", prompt::spec_review(..))
+    worktree.push                   # for commits the session left unpushed
+    run::mark_pr_ready              # open, targets the Base branch, then ready
+  on failure: failed_run::fail      # exit 1, Spec PR stays a draft, URL on stdout
  • The new prompt::spec_review is shaped like the Continuation prompt and uses the existing Factory skills. It asks for /thirdshift:code-review <base> with the Spec as the spec, fixes via /thirdshift:tdd where they fit, a push, an update to the Spec PR through /thirdshift:pr with "Unaddressed findings" and Closes #<spec>, and ends with the headless instructions.
  • The session log is <owner>-<repo>-issue-<spec>-<timestamp>-spec-review.jsonl.
  • The Run and the Spec review share one check that marks the PR ready, run::mark_pr_ready.
  • The prompt is on the Prompts and skills page as "Spec review", and site/prompts/index.html has been regenerated.

Evidence

  • Before: The Spec PR opened ready for review as soon as the last Ticket landed, and no agent ever looked at the Spec branch as a whole.
    After: The new tests in tests/spec_run.rs pass. cargo test passes in full and cargo clippy --all-targets is clean.
once_the_last_ticket_lands_the_spec_review_reviews_the_spec_branch_against_the_base_branch
  sessions == [#21, #22, #20]; the Spec review runs on issue-20
  its prompt names the Spec URL, "main as the fixed point", Unaddressed findings, Closes #20
the_spec_reviews_commits_reach_the_spec_branch_on_origin          (agent pushes)
a_spec_review_that_commits_without_pushing_still_reaches_origin   (thirdshift pushes)
the_spec_pr_is_a_draft_during_the_spec_review_and_marked_ready_after_it
the_spec_review_writes_the_spec_pr_body                           (thirdshift doesn't overwrite it)
the_spec_review_is_logged_under_the_specs_name
a_failed_spec_review_ends_the_spec_run_without_marking_the_spec_pr_ready
  exit 1, Spec PR still a draft, its URL on stdout, session log reported
a_spec_pr_closed_during_the_spec_review_fails_the_spec_run
prompts_page_shows_what_the_binary_sends                           (golden file)

Merge Danger

Door: two-way

This only changes code, with no stored state or migration, so a revert takes it back.

Blast Radius: Spec runs

Only Spec runs change: they now run one more agent session, and the Spec PR is a draft until that session ends. Runs on an issue with no sub-issues behave as before. The only change on their path is that the PR check moved into run::mark_pr_ready, with the same behaviour.

Unaddressed findings

Standards

  • create_pr renamed to create_draft_pr rather than given a draft flag. It has one caller, which always wants a draft, so a flag would be Speculative Generality.
  • ADR-0006 says the Spec PR goes through "the usual Repair loop", and this PR doesn't do that. The Spec PR's Repair loop and Self-merge are The Spec PR's Repair loop and Self-merge #146.

Spec

  • The draft Spec PR opens only after the last Ticket lands, not after the first. Opening it after the first Ticket, with its Tickets checklist, is The draft Spec PR and its Tickets checklist #147. This PR only makes the Spec PR a draft until the Spec review ends.
  • mark_pr_ready also fails if the Spec PR targets a branch other than the Base branch. The issue only asks for "exists and is open". The extra check comes from sharing one check with the Run, and it guards the right thing.
  • A failed Ticket still ends the Spec run straight away, with no draft Spec PR. Failed spec runs are Going as far as it can: Failed spec runs #144.
  • The sessions_by_issue test helper relies on the Spec URL being the first Issue URL in the prompt. That holds for every prompt today, and the fake claude picks its scripts by the same rule.

Closes #142

🤖 Generated with Claude Code

JacobStephens2 and others added 3 commits September 30, 2026 00:26
Once every Ticket has landed, the Spec run fast-forwards the Spec branch from
origin, opens the Spec PR as a draft, and starts a Spec review session in the
Spec branch worktree, logged under the Spec's name. Its prompt, shaped like the
Continuation prompt, has /thirdshift:code-review review the Spec branch against
the Base branch and the Spec, fix what it agrees with, and update the Spec PR
through /thirdshift:pr with Unaddressed findings and Closes #<spec>. thirdshift
then pushes the Spec branch, checks the Spec PR is open and targets the Base
branch, and marks it ready. A failed Spec review ends the Spec run non-zero
with the Spec PR left a draft. The prompt is on the Prompts and skills page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address review findings: a failed Spec review now commits and pushes its work
and reports the draft Spec PR, as a failed Run does, with the log only if it
was written. The Run and the Spec review share one check that marks the PR
ready, and the Spec review prompt names the Spec on its first line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JacobStephens2
JacobStephens2 merged commit 05d415b into main Sep 30, 2026
13 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-142 branch September 30, 2026 00:50
@JacobStephens2 JacobStephens2 mentioned this pull request Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spec review before the Spec PR is ready

1 participant