Skip to content

Deepen Trial execution and fixed-Bar recovery - #1310

Merged
JacobStephens2 merged 2 commits into
mainfrom
issue-1309
Oct 25, 2026
Merged

JacobStephens2 merged 2 commits into
mainfrom
issue-1309

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

Give the Trial module one complete operation and apply the saved Bar's retry rules during recovery.

workers: alternating Arm queue, two workers, dispatch gates
  Trial Runner.run(Eval task, Arm, index)
    resume pending history under the saved Bar
    execute a fresh attempt when eligible
      publish attempt → pending history → terminal result
      keep Trace and active evidence live through publication
    checkpoint before pausing; never replay a retained pause

Closes #1309

Evidence

  • Before: cargo test --test experiments retained_pending_history_obeys_the_saved_void_cause_list -- --exact failed: resume launched 4 attempts when the saved Bar allowed only the 2 retained attempts.
    After: the same regression passes with 2 attempts and retained terminal Void results.
  • cargo check --all-targets passed.
  • cargo test --bin thirdshift evals::experiment:: passed: 49 tests, including complete Trial recovery/publication, execution cleanup, retained readers, Model Scores and rechecks.
  • cargo fmt --check and git diff --check passed.
  • scripts/affected-tests main passed: all 50 site tests and the full Rust suite, including 80 Experiment command regressions (one fixture entry ignored). It used the documented cargo test fallback because nextest is unavailable.
  • /usr/bin/time -p scripts/affected-tests main: 1420.54 seconds wall time (23m 40.54s), including site checks and 40.99 seconds of Rust compilation. Output is retained as .thirdshift-review-CxJABh/affected-tests.log.
  • CI passed on 8b6d7ad5, including Linux tests, the macOS gate, site and Console checks.

Test seams

  • Complete prepared Trial runner, with scripted scoped execution and real temporary storage, observed through retained Trial results and pending histories.
  • Single-attempt execution completion callback, including evidence lifetime and cleanup on callback errors.
  • Experiment CLI commands, including fixed-Bar resume and two-worker scheduling.
  • Existing retained-result reading, Model Score application and recheck interfaces.

Review coverage

Independent Standards and Spec reviews against main each found 0 issues and read all 13 changed files. Unread changed files: none. Generated-output exclusions: none; both reviewers inspected the generated documentation hunk. Reports are retained with this session as .thirdshift-review-CxJABh/standards.md and .thirdshift-review-CxJABh/spec.md.

Unaddressed findings

  • Standards: none.
  • Spec: none.
  • Security: none.

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

Internal ownership changes preserve the retained format, filenames and Experiment commands. Recovery now obeys older saved cause lists and cannot gain extra attempts after terminal-publication failure.

Blast Radius: Experiments

Retry, pause and publication behavior for Trials; ordinary Ticket runs are unaffected.

Built with codex · gpt-6.1-sol · xhigh

@JacobStephens2
JacobStephens2 merged commit e48adc9 into main Oct 25, 2026
18 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1309 branch October 25, 2026 10:12
@JacobStephens2 JacobStephens2 mentioned this pull request Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deepen Trial retry and retained publication behind one complete interface

1 participant