Skip to content

Build blind packets for sampled pull requests - #1139

Merged
JacobStephens2 merged 4 commits into
issue-978from
issue-1025
Oct 23, 2026
Merged

JacobStephens2 merged 4 commits into
issue-978from
issue-1025

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

Each sampled pull request now gets a blind packet before its Scoring window is saved.

sampled pull request
  issue edit history → title/body at merge
  captured PR base → merge commit → complete merged diff
  merge commit → repository files (Git metadata and attribution removed)
  pull request → title/body with attribution lines removed
  CI → check names and conclusions
  private packet → saved window

The sample retains Harness attribution separately. Missing historical issue text fails the Command without advancing its window. Grader sessions remain separate work.

Closes #1025

Evidence

  • Before: cargo test --test scoring packets_are_blind_and_keep_the_issue_and_repository_at_merge failed at saved packet directory.
    After: The same test passes for Claude and Codex packets, including removal of commit messages and exclusion of a later fix.

  • Before review fixes: the reviewers' exact commands failed with first PR commit missing, packet repository leaked Built with, and blinding rewrote a binary repository blob.
    After: each exact regression passes and stays in tests/scoring.rs:

    • cargo test --test scoring packet_diff_includes_every_commit_in_a_rebased_pull_request -- --exact --nocapture
    • cargo test --test scoring packet_repository_removes_attribution_for_every_harness -- --exact --nocapture
    • cargo test --test scoring packet_blinding_preserves_binary_files_with_valid_utf8_bytes -- --exact --nocapture
  • Focused Scoring coverage also checks paginated history, deleted revisions, private storage, source literals, binary files, symlinks, executable bits and exclusion of earlier base changes.

  • Before the final fidelity fix: cargo test --test scoring packet_diff_preserves_trailing_whitespace_in_the_last_changed_line -- --exact --nocapture showed the final diff line losing its spaces.
    After: the same test passes; cargo test --test scoring passes all 19 tests (one fixture entry point ignored).

  • cargo check and cargo clippy --all-targets -- -D warnings: passed.

  • scripts/affected-tests 9ab127320007357c695c5f90593f7417c2312286: site checks passed (43 tests). Broad Rust validation is reused from successful CI on final head 3296899, as docs/testing.md permits. Full Rust tests, formatting, Clippy and both browser jobs passed.

  • Timing: /usr/bin/time -p cargo nextest run reached 985.92 seconds wall time, compilation included, before the duplicate local run was stopped after CI passed its head. This is a partial timing, not a completed full-suite benchmark.

Test seams

  • thirdshift score / thirdshift scores --json, with fake gh edit history and CI and a real local Git origin; inspect the saved Grader packet and repository snapshot.

Unaddressed findings

Standards

None. The one maintainability finding was addressed by sharing Issue-branch inference.

Spec

None. All three behavioral findings were reproduced with the reviewers' tests, fixed and kept as regressions; no decision was deferred.

Security

None. Guidance-mode review of git diff 9ab127320007357c695c5f90593f7417c2312286...HEAD plus supporting code found no introduced finding with a demonstrable cross-boundary impact: git/gh run via argv with no shell, merge/base commits validated to 40 hex, branch-derived issue numbers are digits-only with strict IssueUrl parsing, the fetch host is fixed to github.com, symlinks are never followed, NUL-bearing blobs are preserved, and packet storage is 0700. No proof-of-concept failed, so no fix and no new regression test. cargo test --test scoring passes 19 tests.

Security review outcome

No unaddressed introduced findings.

Review coverage

Standards and Spec ran independently against fetched issue-978, pinned to 9ab127320007357c695c5f90593f7417c2312286, at e37266e and followed up at 77da798. Both axes read all 13 changed files, including the generated site/docs/index.html hunks. No generated files were excluded, and no changed file was left unread. The parent then closed the follow-up binary finding and the final whitespace preservation check in src/scoring/packet.rs and tests/scoring.rs, retaining both regressions.

Merge Danger

Door: two-way

Rollback stops packet generation; existing sample files remain readable.

Blast Radius: Scoring

Scoring now reads issue history and CI and fetches each sampled merge commit. Failure in packet construction fails that Command before saving its window.

Built with codex · gpt-6.1-sol · xhigh

@JacobStephens2
JacobStephens2 merged commit 8905eb4 into issue-978 Oct 23, 2026
18 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1025 branch October 23, 2026 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant