[JacobStephens2]

Summary

score
  finish pending packets from earlier windows
  sample merged pull requests → blind packets
  choose independent [scoring] Harness + Model
  sandbox Grader → validate five Scores → checkpoint each pull request
  plan-limit warning → stop; leave remainder for next score

Scores retain the Run's Built-with attribution, the Grader's Harness/Model and a local Command-record link when available. A blank Grader Model fails before launch because independence cannot be established. The new Session prompt and site documentation are regenerated.

Backports the model-session sandbox/proxy and Claude/Codex limit detection from #1014/#1008 on main, which issue-978 does not yet contain. No Cargo dependency changes.

Closes #1026

Evidence

• Before: cargo test --test scoring grading::an_unknown_default -- --nocapture failed: an unspecified default Model was allowed to grade its own possible output.
  After: the same regression passes, requiring an explicit independent Model before launch.
• Fake Harness tests cover all five Scores/reasons, fallback selection, two matching Models, malformed Scores, retained attribution and local Command links, Claude warning/rejection and live Codex native quota reports, and carry-over without repeating completed Scores.
• The real bwrap test passes: host secrets and sibling packets are invisible, and repository writes fail.
• Before: cargo test --test scoring sandboxed_grader_cannot_read_ambient_secrets_or_replace_a_live_login -- --nocapture failed with exit 101. The Grader read the operator's ambient RESEND_API_KEY and ~/.cargo/credentials.toml, and replaced the live Codex login.
  After: the same test passes. The sandbox clears the environment, copies the login, and hides Cargo registry credentials. The operator's login file is unchanged. All 26 scoring integration tests pass, and cargo clippy --all-targets -- -D warnings passes.
• Final validation: RUST_TEST_THREADS=8 scripts/affected-tests issue-978 passed: 43 site browser tests and the full Rust suite, including 1,256 unit tests and all 25 Scoring integration tests. cargo clippy --all-targets -- -D warnings and the prompt regeneration/check also passed. Affected-check wall time was 1,474.19 seconds, including browser tests and cached Rust compilation; nextest was unavailable, so the selector used cargo test.
• The first broad run overlapped a Cargo rebuild, replacing the executable used by re-exec fixtures and causing five ENOENT failures. A deleted-executable reproduction produced the identical failure; a stable-executable control passed. Final validation runs without overlapping builds.

Seams under test

• Whole binary against fake GitHub and Harness executables: saved Scores and attribution, independent Model selection, packet isolation, and plan-limit carry-over as exposed by score and scores --json.

Review

• Standards: one P3 design judgment addressed by removing unused process hooks; no unresolved findings.
• Spec: no findings.
• Both reviewers read all 35 changed files, including generated hunks. Unread changed files: none. Generated-output exclusions: none.
• Final reports: .thirdshift-review-raUBY6/standards.md and .thirdshift-review-raUBY6/spec.md, retained with the Session.

Unaddressed findings

Standards

None. The single reported judgment was addressed.

Spec

None. No findings were skipped or deferred.

Security

None. The Grader sandbox isolation gap is fixed in src/sandbox.rs and kept by sandboxed_grader_cannot_read_ambient_secrets_or_replace_a_live_login.

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

New optional Score fields preserve existing samples. Pending packets survive errors and plan-limit stops.

Blast Radius: scoring

Also backports shared process supervision and quota interpretation; ordinary Runs do not opt into plan-limit stopping. Production grading requires bwrap/socat and an explicit Model. Other Harnesses await verified warning protocols, as #1008 specifies.

Built with codex · gpt-6.1-sol · xhigh