Skip to content

Review run: review another author's pull request and address its findings - #1718

Merged
JacobStephens2 merged 29 commits into
mainfrom
issue-706
Nov 4, 2026
Merged

JacobStephens2 merged 29 commits into
mainfrom
issue-706

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 4, 2026 •

Copy link
Copy Markdown
Owner

Summary

Add Review runs for another Trusted account's pull requests, started by URL or as a one-PR author Pass.

review <PR URL> | review --author <login>
  check eligibility, Command limit and origin reservation
  capture PR head/base and prepare trusted Spec input
  review Standards + Spec, address findings unless --recommend-only
  run repository validation and optional Security review
  validate the final head after later fixes
  deliver fixes without rewriting history and post one review comment
  record reviewed/pushed heads, notify, and wait in the Inbox

The Process catalogue also supports pull-request worktrees for Custom processes. The command, configuration, generated prompts and user documentation describe the same workflow.

Review runs refuse forks, drafts, closed/merged pull requests and authors who are not Trusted accounts. Linked closing issues supply the Spec; the comment explicitly identifies body-only reviews. A departure justified only by the PR body remains an Unaddressed finding for the Day shift, without a triage issue.

The author Pass skips heads recorded by thirdshift's acting account and waits for open bases to be reviewed. It warns an upper PR when recorded base fixes are missing from its branch, leaving stack updates to the author. A moved head withholds fixes while preserving the findings comment, including when Security review is enabled.

Repository instructions and review.commands_by_repo supply validation; worktree-invalid suites are reported as not run against the branch. Later editing Steps trigger validation of the final head. Optional Security review uses the configured Harness and private records; public validation text that copies private report fields or lines is withheld.

Closes #706

Evidence

  • Before: review machinery ran inside thirdshift's own Runs. After: review <PR URL> and review --author <login> support a colleague's PR, with --recommend-only for findings without branch changes.
  • Each review regression failed with exit 101 before its fix and passed with exit 0 afterward:
    • cargo test --test spec_review_706_regressions security_requested_review_reports_findings_when_the_author_moves_the_head -- --exact --nocapture
    • cargo test --test spec_review_706_regressions author_pass_ignores_a_review_marker_from_a_different_trusted_account -- --exact --nocapture
    • cargo test --test spec_review_706_regressions upper_stack_comment_warns_when_the_base_received_review_fixes -- --exact --nocapture
    • cargo test --test review_validation_after_security validation_does_not_publish_a_copied_private_description_line -- --exact
  • scripts/affected-tests main: generation checks and all 61 site tests passed. Its Rust step initially hit the host's file-watch quota; the three affected Console cases passed after temporarily raising the quota. The original setting was restored.
  • Full Rust rerun: 3,675 passed, 5 failed, 80 skipped, with three test workers (3,541.14 seconds including a warm build). Four failures were systemd scope-name collisions; one was the OTLP timing assertion. All five passed individually afterward. This broad run preceded the final repository-name case normalization in the stack lookup.
  • Final Review selection on the latest tree: 44 passed, 5 skipped, covering URL runs, author Passes, optional Security review, validation after Security fixes, and the retained regressions. cargo fmt --check and the staged-diff whitespace check passed.

Test seams

Exercise the real thirdshift review CLI against real Git repositories and the existing external GitHub/Harness fakes. Assert on origin commits, posted review comments, selection, labels, Command records and notifications. Keep regression tests at this existing Scenario seam.

Unaddressed findings

Standards

None. The reviewer raised one finding: a copied private description line could reach the public validation comment. Its regression reproduced the disclosure; the fix now withholds that text. The follow-up raised no new findings.

Spec

None. The reviewer raised three findings: Security-enabled moved-head reporting, acting-account marker ownership, and upper-stack update warnings. Each reproduced, was fixed, and passed its regression. The follow-up raised no new findings. No finding was declined or deferred to the Day shift.

Security

None. The reviewer raised two findings: the posted review report could include a prior private Security report, and a marker inside the session report could record a head that was not reviewed. Each reproduced with a failing test, was fixed, and its regression now passes. No finding was left unaddressed.

Security review outcome

No unaddressed introduced findings.

Review coverage

Both reviewers read all 59 original changed files and the new regression file, plus supporting standards, domain docs and ADRs. Generated prompt/manual output and affected hand-written HTML were inspected directly. No generated-output exclusions and no unread changed files remain on either axis. Reports record the final files-read lists; finding counts are Standards 1, Spec 3, including findings fixed during review.

Merge Danger

Door: two-way

The code and configuration can be reverted; comments and commits already delivered to colleagues' branches remain visible.

Blast Radius: repository

Review runs can append commits and comments to eligible same-repository pull requests. Forks, drafts, closed/merged requests and untrusted authors are refused.

Built with codex codex-cli 0.164.0 · gpt-6.1-sol · xhigh

Tickets

A pull request can commit .thirdshift-review-input.md as a symlink. Writing the review packet followed that link and replaced a file outside the worktree, then continued the review. Refuse the symlink before a session starts.
The new review-run Built-in process is included in the compiled catalog, but the listing test still expected only the previous four built-ins. Include its name and description in both exact sorted-output assertions.
Add recommend-only pull request Review runs
A plain fast-forward restored commits removed from the pull request branch when its tip was an ancestor of the reviewed head. Compare that head with a lease so publication stays a fast-forward only while the branch is unchanged.
Address Review run findings on the pull request branch
Run requested Security reviews during Review runs
Review one author’s waiting pull requests as a Pass
Delivery posts the validation session's final message after a Security review has written its private report. Keep that message off the pull request when it contains a stored report path or identifying field.
Preserve Review validation alongside the author Review Pass in the domain
index and hand-written site pages. Regenerate the manual from merged sources.

Validation:
- cargo clippy --all-targets -- -D warnings and cargo fmt --all -- --check pass.
- scripts/affected-tests origin/issue-706: all 61 site tests pass; Rust stops
  after the shared user's file-watch quota prevents three Console tests.
- cargo nextest run --no-fail-fast, three threads and Console tests under a
  separate UID: 3668 passed, 7 failed, 79 skipped. Wall time 3602.03s includes
  the 0.75s cached test build. All seven failures pass exact targeted reruns:
  six file-watch cases under the separate UID and one systemd scope collision
  under the normal user. The full run is not counted as an unqualified pass.
Run repository-specific validation in Review runs
Preserve moved-head findings through later Steps, recognize markers only from the acting account, warn upper pull requests about missing base fixes, and withhold private Security report excerpts. Keep the reproduced regressions and update the manual and hand-written site pages.
@JacobStephens2
JacobStephens2 marked this pull request as ready for review November 4, 2026 09:31
Preserve Review and Triage runs in shared command plumbing and documentation. Regenerate the manual and prompts, retain repository-key validation, and initialize the new optional Triage result for Review runs.
@JacobStephens2
JacobStephens2 merged commit f942dd9 into main Nov 4, 2026
16 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-706 branch November 4, 2026 10:05
@JacobStephens2 JacobStephens2 mentioned this pull request Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Review run: review another author's pull request and address its findings

1 participant