[JacobStephens2]

Summary

Spec-review imports now copy the original issue/PR URLs and Base/Spec branch names into [trial]. Reruns fill missing tables and preserve edited tables and other local task edits.

 evals import-replays
+  Spec manifest URLs and branches → missing Trial inputs
 new Experiment
+  name approved tasks missing [trial] before either Arm is built
   probe/build Arms → Trials → report grading
 saved Experiment
   reuse recorded inputs

Spec-review Trials keep the production prompt. Their sandbox has no PR service: gh calls fail and only the reports are graded. Updated the protocol, manual and hand-written site pages.

Closes #1251

Evidence

• Before: cargo test --test evals replay_import_completes_trial_inputs_and_preserves_local_edits failed because the Spec-review table was absent. After: all 14 Eval CLI tests pass, including exact manifest values, legacy backfill and preservation of edited tables.
• Before: the ordinary Experiment regression reported no task names; the release regression reached fake Cargo before validation. After: both modes name every missing approved task, ignore proposals and never reach an Arm build.
• Before: the Spec reviewer's exact cargo test --test experiments spec_review_completed_experiment_uses_saved_inputs -- --exact failed when a completed Experiment's live table was removed. After: the retained regression passes; it now covers both ordinary and release invocations. The Experiment file passes: 69 passed, one fixture entry point ignored.
• The fake-Harness Spec-review Trial retains both reports after a failed gh pr view and still receives the production PR-edit prompt.
• Final validation: cargo fmt --check, cargo check, and /usr/bin/time -p scripts/affected-tests issue-977 passed on a869d341. The selector passed 44 site tests and 2,566 Rust tests (58 skipped). Nextest reported 2,399.697 seconds; the complete selector took 2,481.77 seconds wall time, including 18.72 seconds of Rust compilation and the site checks.
• An earlier broad run had three Console connection failures while a concurrent regression build replaced the watched executable. With builds sequenced, the same three checks passed (3/3).

Test seams

• evals import-replays CLI with made-up replay manifests and local edits.
• experiment CLI with cold Arm caches, fake Cargo and saved Experiments.
• --trial JSON protocol with a fake Harness and an unavailable PR service.

Review

• Fixed point: issue-977 (298b6b8b9cc60b876dce971dd027ce60b6f27155).
• Standards: 0 findings. Spec: 1 P2 reproduced with the reviewer's exact test, fixed, and retained as a regression; no additional findings on follow-up.
• Both axes read all 14 changed files, including generated site/docs/index.html. No generated-output exclusions. Unread changed files: none.

Unaddressed findings

Standards

None.

Spec

None.

Security

None.

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

Blast Radius: Evals

Changes affect replay import and new Experiment preparation. Existing edited Trial tables and saved Experiment inputs are preserved.

Built with codex · gpt-6.1-sol · xhigh