Skip to content

Deepen Trial execution through completed mechanical Scores - #1292

Merged
JacobStephens2 merged 2 commits into
mainfrom
issue-1290
Oct 25, 2026
Merged

JacobStephens2 merged 2 commits into
mainfrom
issue-1290

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

workers
  prepare fixed Harness directories once
  execute selected Trial
    fresh repository → launch → supervise → classify → mechanical Score
    callback: checkpoint → pause/exhaustion publication
    finalize Trace → remove active evidence
  retry or publish ordinary terminal result

Trial execution owns execution ordering and keeps its guards private through the worker's callback. Workers retain the queue, Writer and fixed Bar policy. The domain-to-code index names both owners.

Closes #1290

Evidence

  • Before: the new scoped-interface test failed with Trial execution is not implemented.
    After: all nine focused execution tests pass, covering completed Scores, fallback/mutation, timeout cleanup, unexplained kills/cap precedence, raw/trace native limits, interruption, sandbox rescoring and callback/error lifetime.
  • cargo check, cargo fmt --check and cargo clippy --all-targets -- -D warnings pass locally.
  • CI on final commit 32a97512 passes: full Linux cargo test, macOS gate, Console and site checks. Its log confirms all nine new execution tests pass on that exact commit.
  • scripts/affected-tests main selected the full Rust suite (2,834 tests across 63 binaries). Per docs/testing.md's same-head CI reuse rule, the redundant local run was stopped after the exact-head full CI suite passed; it is not reported as a completed local pass.
  • Environment skips: this host has no reachable systemd user manager. Direct runs confirmed the two live Trial memory-cap tests and the sandboxed Code Grader cap test skip for that reason. Fixture-backed cap classification and sandbox-unavailable rescoring pass. The runner also selected its existing 60 ignored tests as skipped.

Test seams

  • execution::Execution::prepare and Execution::run: one selected Trial with real temporary Git repositories and executable Arm/system-manager fixtures, including completed Scores and retained evidence lifetime.
  • Existing Experiment CLI regressions: scheduling, retry budgets, Stop/resume, accounting, Trace export, sandbox startup and live memory caps.

Review coverage

  • Standards: one advisory duplicate Arm-selection finding addressed by selecting build and setup together; reviewer confirmed the correction.
  • Spec: no findings; reviewer also checked the correction.
  • Both reviewers read every changed file, including the relocated monitor and its predecessor. No generated exclusions. No changed file was left unread.
  • Final reports: .thirdshift-review-aLQJ0z/standards.md and .thirdshift-review-aLQJ0z/spec.md in the Session evidence.

Unaddressed findings

  • Standards: none.
  • Spec: none.
  • Security: none. Focused guidance-mode review of the 4ed03e7...HEAD diff: the execution body moved verbatim (relocated monitor differs by one doc-comment line only), the grading timeout value is unchanged, and arm/setup selection stays consistent; all nine execution tests pass. No introduced finding; no PoC, no fix.

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

Reversible internal ownership change; retained version-1 requests/results and Experiment data stay compatible.

Blast Radius: Trials

Execution ordering and evidence lifetime affect Experiment Trials; scheduling, retry budgets and publication remain worker-owned.

Built with codex · gpt-6.1-sol · xhigh

@JacobStephens2
JacobStephens2 merged commit bd2dac6 into main Oct 25, 2026
18 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1290 branch October 25, 2026 01:58
@JacobStephens2 JacobStephens2 mentioned this pull request Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deepen Trial execution through completed mechanical Scores

1 participant