Skip to content

Evidence-backed review findings - #496

Merged
JacobStephens2 merged 18 commits into
mainfrom
issue-486
Oct 7, 2026
Merged

JacobStephens2 merged 18 commits into
mainfrom
issue-486

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Summary

Closes #486

Every Session that runs the code review now uses the same evidence, decline, and coverage rules: fresh and Continuation implement Sessions, the Spec review, and the Repair that reviews Foreign commits.

Session prompt → shared review instruction + git-ignored report paths
  Standards + Spec reviewers
    correctness findings → written-out test or command
    files read → one follow-up over skipped changed files
    final findings + coverage → standards.md / spec.md
  author runs evidence as written before deciding
    confirmed failure → fix + retained test
    decline → refuting evidence, or linked decision issue
  Sessions keeps opaque reports beside its Session log, Resume included

Spec findings keep quoting requirements; Standards findings keep citing standards or naming smells. Written-out evidence and files-read lists sit outside the expanded prose limit. A passing run supports a decline only when it exercises that particular finding. Each decline cites a line of code, plan decision, ADR, or that run. Calls left to a Spec or the Day shift link an existing open issue or create one carrying only needs-triage, with the finding, evidence, originating PR, and reason for the decision. Remaining unread changed files are named in the review and PR body.

Sessions supplies report paths on every Harness and retains whichever reports exist under the Session log's name with .standards and .spec extensions, including after a failed Session. Missing reports and setup or retention errors produce progress lines and preserve the Session outcome. Generated prompts, the prompts page, and CONTEXT.md describe the same contract. The final Standards review also consolidated duplicated execution-test setup.

Evidence

  • Before: main asks authors to address findings they agree with and give skipped findings a one-line reason; it has no shared reviewer-report retention.
    After: The generated prompts/page checks verify the evidence, decline, coverage, and report-path instructions. Execution tests verify opaque report retention across all six Harnesses, Resume retention, failed-Session retention, and continuation when reports are missing or setup fails.
  • Before: The reviewed head f3a0302 repeats the review execution fixture setup in the normal path and the report-setup failure regression.
    After: Both use one helper; the existing behavior assertions remain intact.

Validation:

  • cargo test on the reviewed head f3a0302: 1,431 passed, 1 existing ignored test.
  • After consolidating the execution-test fixture: cargo test --bin thirdshift session::execution_tests: all 9 passed.
  • cargo fmt --check, cargo clippy --all-targets -- -D warnings, and git diff --check: passed after the refactor.

Seams under test

  • Sessions through Sessions::within, with fake Harness CLIs and observable prompts, retained reports, progress, and Run outcomes.
  • Session prompts through the existing generated prompts-page and prompt-file checks.

The skill's text was reviewed directly, as #486 specifies.

Unaddressed findings

Standards

None. The single finding, duplicated execution-test setup, was addressed.

Spec

None. The review checked #486 as a whole, including the integration of all four Tickets.

Both reviewers read every changed file; none remain unread.

Files read by both Standards and Spec reviewers
CONTEXT.md
prompts/continuation-pr.md
prompts/continuation.md
prompts/fresh.md
prompts/review-repair.md
prompts/spec-review.md
site/prompts/index.html
skills/thirdshift-code-review/SKILL.md
src/prompt.rs
src/prompts_page.rs
src/session.rs
src/session/execution_tests.rs
src/session/review_reports.rs
tests/fresh_run.rs
tests/merge_run.rs
tests/support/mod.rs

Merge Danger

Door: two-way

The review instructions and report storage can be reverted without a data migration.

Blast Radius: Sessions

The review contract applies to implement Sessions, Spec reviews, and Foreign-commit Repairs on every Harness. thirdshift keeps reports without interpreting them or gating delivery on findings; Failed-run rules, the Repair loop, the Claim, and the Self-merge retain their existing behavior.

Tickets

Built with codex · gpt-6.1-sol · xhigh

@JacobStephens2
JacobStephens2 marked this pull request as ready for review October 7, 2026 13:15
…g to fix on the branch)

2026-10-07T13:45:17Z, host thirdshift. Uncommitted work at the time of failure is included in this commit.
@JacobStephens2
JacobStephens2 marked this pull request as draft October 7, 2026 13:45
@JacobStephens2
JacobStephens2 marked this pull request as ready for review October 7, 2026 14:47
@JacobStephens2
JacobStephens2 merged commit 200661b into main Oct 7, 2026
13 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-486 branch October 7, 2026 15:18
@JacobStephens2 JacobStephens2 mentioned this pull request Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evidence-backed review findings

1 participant