[JacobStephens2]

Summary

Experiment
  run both Arms' Trials
  apply mechanical checks
  shuffle and blind one packet per Eval task
  run a fresh Grader on another configured Model when available
  validate and save each Trial's Score
  carry wrong findings and borderline marks into the Verdict

Closes #1005

Evidence

Test seams: the experiment and experiments --json commands with fake Harnesses, the Grader packet/Score file contract, and private Grader Session artifacts.

• Before: cargo test --test experiments model_graders::metadata_rows_beneath_generic_table_headers_are_blinded -- --exact failed: the packet exposed baseline, candidate, Harness, Model and Effort values. cargo test --test experiments model_graders::model_grader_sessions_keep_otlp_traces -- --exact failed because the Grader had no OTLP Session trace.
  After: Both exact reproductions passed. All 10 model Grader tests passed on the final head, including private-output retention and timeout/resume without rerunning Trials. Genuine finding tables remain intact.
• Follow-up reproduction cargo test --test experiments model_graders::metadata_rows_beneath_setting_value_headers_are_blinded -- --exact also failed before the correction and passed afterward. Metadata first-column headings now identify metadata independently of the second column's wording.
• Merged the current issue-977, preserving Code review Trials, Void Trial rules and plan-limit handling. cargo check, cargo clippy --all-targets -- -D warnings, and the 12 prompt generation tests passed.
• /usr/bin/time -p scripts/affected-tests issue-977 passed: all 43 site checks and the full Rust suite (cargo test fallback). Wall time: 1,950.45 seconds (32m 30s), including compilation and browser checks. The full run includes both metadata regressions and the Grader trace regression.
• Final-head cargo check, cargo clippy --all-targets -- -D warnings, cargo fmt --check, and cargo test --test experiments model_graders:: passed. CI passed on a28f723.
• Security review: cargo test --test experiments model_graders::metadata_rows_contiguous_with_a_findings_table_are_blinded -- --exact failed before the fix (packet leaked Harness/Model values through metadata rows contiguous with a findings table) and passed afterward; all 11 model Grader tests, cargo clippy --all-targets -- -D warnings, and cargo fmt --check passed on the final head.
• Repair validation (2026-10-23): 43 site checks, 1,254 Rust unit tests and all 61 integration targets have passing local results across scripts/affected-tests issue-977 and follow-up runs of the full release suite plus the 20 targets the selector had not reached. The selector itself stopped on the unrelated release fixture race documented in CI notes.

Unaddressed findings

Standards

None. Addressed the missing Session traces and repeated Grader fixture parsing.

Spec

None. Addressed metadata identities leaking beneath generic table headings.

Security

None. One introduced blinding bypass (metadata rows contiguous with a findings table reached the Grader packet) was fixed with a regression test; no unaddressed introduced findings remain.

Review coverage: Standards 25/25 changed files read; Spec 25/25 read. No generated-output exclusions and no unread changed files. Both reports, including their final files-read lists, are retained in .thirdshift-review-ZhJNLM/standards.md and .thirdshift-review-ZhJNLM/spec.md in the worktree.

Security review outcome

No unaddressed introduced findings.

CI notes

• Failed test job: console_install::unit_preserves_literal_paths_with_shell_and_systemd_metacharacters intermittently fails at tests/console_install.rs:404 with ETXTBSY before application startup when a parallel fixture child retains fs::copy's writable descriptor; test, installer and CI workflow are unchanged from issue-977@32c12af, local repetition reproduced it after 78 passes, and the full-suite run subsequently passed it.
• Local validation also hit ETXTBSY at tests/release.rs:304 in main_moving_during_ci_is_followed_and_the_bump_uses_the_tested_tip: its in-process-written gh fixture has the same race, and the test/support are unchanged from issue-977@32c12af; its exact test and full release suite passed afterward.

Merge Danger

Door: two-way

Blast Radius: Sessions

Written criteria now require model grading. Shared Session execution handles Grader deadlines and private output. Graders run without a sandbox until #1014.

Built with codex · gpt-6.1-sol · xhigh