Skip to content

Complete Trial repository preparation under isolated Git policy - #1300

Merged
JacobStephens2 merged 4 commits into
mainfrom
issue-1297
Oct 25, 2026
Merged

JacobStephens2 merged 4 commits into
mainfrom
issue-1297

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

Complete Trial repositories under preparation's isolated Git policy before Harness setup and integrity capture. Inspection remains a detached base with the reviewed patch staged.

Trial execution
  complete repository preparation
    import and verify base-only history
    create requested branches and commit staged reviewed patch
    publish Spec-review refs and exclude issue input
    publish complete repository
  Harness instruction fallback
  integrity snapshot
  launch Arm

Closes #1297

Test seams

  • Eval repository preparation: real local Git repositories through inspection and complete Trial preparation, checking refs, synthetic trees, input overlays, history confinement, origin publication, hooks and failure cleanup.
  • Trial execution: the existing isolated execution fixture, checking caller Git hooks cannot alter Trial inputs and preparation failure launches no Arm.
  • Existing Eval and Experiment CLI regressions through the repository's affected-tests entry point.

Evidence

  • Before: cargo test --bin thirdshift trial_preparation_ignores_caller_checkout_and_push_hooks -- --nocapture failed with post-checkout ran.
    After: passes with checkout and push hooks installed through the caller's global Git configuration after source fixture construction.
  • Before: the Spec review's exact regression command, cargo test --bin thirdshift evals::repository::tests::spec_review_accepts_branch_below_base_seed -- --exact --nocapture, failed because the conventional origin seed blocked base/issue-7.
    After: passes; complete Trial preparation seeds the requested Base branch. The regression is retained, and the contract matrix covers both base/stable and base/issue-7, plus an Issue branch named base.
  • Repository contract tests cover synthetic commits, copied input overlays, absent source commits, final origin refs and pushes, invalid inputs, failure cleanup and existing destinations. The execution regression confirms preparation failure publishes no repository and launches no Arm.
  • cargo check, cargo fmt --check and git diff --check passed.
  • /usr/bin/time -p scripts/affected-tests main passed: all 50 site checks and the full Rust suite (cargo test; nextest unavailable). Wall time: 1,419.82 seconds (23m 40s), including compilation and browser checks. The binary suite passed 1,431 tests with three ignored; the Eval CLI and all 79 Experiment integration tests passed.

Review coverage

Standards: 0 findings. Spec: 1 P2 finding, reproduced and fixed. Both follow-up reviews cover the final changes. All seven changed files were read by both axes; no unread changed files or generated-output exclusions remain.

Merge Danger

Door: two-way

Blast Radius: Evals

Changes affect throwaway Trial repository preparation and its Experiment caller. No persistent data migration.

Unaddressed findings

Standards

None.

Spec

None.

Security

None.

Built with codex · gpt-6.1-sol · xhigh

Security review outcome

No unaddressed introduced findings.

@JacobStephens2
JacobStephens2 merged commit 076b066 into main Oct 25, 2026
18 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1297 branch October 25, 2026 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deepen Eval repository preparation through complete Trial setup

1 participant