Skip to content

Count Unaddressed findings by kind in Run metrics - #1146

Merged
JacobStephens2 merged 3 commits into
issue-981from
issue-1034
Oct 23, 2026
Merged

JacobStephens2 merged 3 commits into
issue-981from
issue-1034

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

Completed Runs retain separate Standards, Spec and Security counts in run_metrics.unaddressed_findings.

Delivery completes, after Repairs
  captured PR body + Security review count
    count findings by kind; ignore nested evidence and repeated Security summaries
    save in the Command record
show --json
  return the saved counts, unchanged by later PR edits

A body without the section records zeros; no PR records null. An unreadable body preserves a known Security review count with null Standards and Spec counts.

Closes #1034

Evidence

  • Before: cargo test --test run_metrics a_run_counts_unaddressed_findings_by_kind_from_its_final_pr_body -- --exact failed: null instead of {standards: 2, spec: 1, security: 1}.
    After: passed; coverage also checks final-body edits by Repairs, later Day shift edits, bodies without the section, no PR, older records, Spec runs and Ticket Runs.
  • Before: both Spec reviewer commands reproduced their findings: nested evidence counted one Standards finding instead of two; an unreadable PR body lost a known Security count of two.
    After: both exact commands passed, with their regression tests retained:
    • cargo test --test run_metrics spec_review_nested_evidence_heading_preserves_the_findings_axis -- --exact
    • cargo test --test security_review spec_review_known_security_count_survives_an_unreadable_pr_body -- --exact
  • The Security review count also survives failed summary publication. cargo test --test security_review run_metrics_include_the_security_reviews_count_even_if_its_body_write_fails -- --exact failed before the fix and passed after it.
  • cargo fmt -- --check and cargo clippy --all-targets -- -D warnings passed.
  • Full Rust CI passed on implementation head 13b0cdb; the duplicate local run was stopped after CI passed. After merging the advanced Base branch, cargo check, cargo test --test run_metrics --test review_summary_resume_repro, the Security body-read regression and formatting passed on 68614d0.
  • scripts/affected-tests issue-981: all 44 local site tests passed. Reused the successful full Rust CI run on exact final head 68614d0, following docs/testing.md; stopped the duplicate local Rust run after CI passed. CI's cargo test step took 12m 19s, including compilation and excluding queue time. The same run passed formatting, Clippy, release-generator checks, site and Console browser tests.

Test seams

  • The whole binary against fake Harnesses and fake gh: completed Runs retain Unaddressed findings by kind, and show --json returns those retained metrics.

Review coverage

thirdshift-code-review reviewed against issue-981, initially pinned from origin/issue-981 at 0767f7c9e6f5bd9dff185bfd52eeed825d08e1d6. Both axes read all 14 implementation files and inspected the fixes through 13b0cdb, including generated HTML directly. Generated-output exclusions: none.

The Base branch then advanced through #1144 / #1035. Merge commit 68614d0 preserves its per-session review counts alongside this PR's Unaddressed findings. The implementing agent inspected and tested the resolutions. Each independent reviewer had already used its one permitted follow-up; the following changed files have final merge hunks left unread by both axes:

docs/guide.md
site/architecture/index.html
site/docs/index.html
site/how-it-works/index.html
site/index.html
src/logs/command_record.rs
src/logs/recording.rs
src/logs/run_metrics.rs
tests/run_metrics.rs

The merge also imports these Base-branch revisions, outside the current PR diff, which these reviewers did not inspect after reconciliation:

prompts/continuation-pr.md
prompts/continuation.md
prompts/fresh.md
prompts/review-repair.md
prompts/spec-review.md
site/prompts/index.html
skills/thirdshift-code-review/SKILL.md
src/logs/effects.rs
src/logs/recording/tests.rs
src/prompt.rs
src/session/review_reports.rs
tests/fresh_run.rs
tests/review_summary_resume_repro.rs

Standards: 0 findings. Spec: 2 P2 findings, both reproduced and fixed; 0 outstanding.

Merge Danger

Door: two-way

Blast Radius: Observability

Adds a completion-time PR-body read and retained metrics. Read failures warn and preserve any independently known Security count.

Unaddressed findings

Standards

None.

Spec

None. Both Spec findings were addressed; no findings were skipped or deferred to the Day shift.

Security

None. The Security review of the issue-1034 change against merge base 371e40a found no introduced Security findings, so nothing is unaddressed. The new code only counts Delivery's own PR-body entries into retained metrics: it takes no privileged action on the counts, keeps the independently recorded Security review count with max() so body edits cannot suppress it, and parses with a linear line scan with no regex, no dynamic execution, no secrets, and no new network or file writes. No proof-of-concept test failed, so no fix was committed. Evidence: cargo test --test run_metrics (8 passed), --test security_review (39 passed), --bin thirdshift pull_request:: (32 passed) and delivery:: (68 passed); cargo fmt --check and cargo clippy --all-targets -- -D warnings clean.

Built with codex · gpt-6.1-sol · xhigh

Security review outcome

No unaddressed introduced findings.

@JacobStephens2
JacobStephens2 merged commit d8ef257 into issue-981 Oct 23, 2026
18 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1034 branch October 23, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant