Skip to content

Deepen Trial results behind one Experiment storage interface - #1289

Merged
JacobStephens2 merged 2 commits into
mainfrom
issue-1288
Oct 25, 2026
Merged

JacobStephens2 merged 2 commits into
mainfrom
issue-1288

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

Trial Workers ────────┐
Experiment metadata ─┤
Retained reading ────┼─> Trial results ─> pending history + terminal evidence
Model Grader ────────┘

Trial results now owns compatible filenames, decoding, captures, pending accounting,
publication order and terminal Score updates. Workers retain Bar decisions;
retained reading retains Session evidence discovery.

Closes #1288

Test seams

  • Trial results writer: initialize, checkpoint a completed attempt, drop/resume and finish.
  • Trial results terminal-only reading and retained capture: eligibility, decoding, captured membership, absence, errors and pending accounting.
  • Trial results Model Score application and effective recheck reads: opaque saved targets, raw evidence isolation and unchanged mechanical accounting.
  • Existing Experiment CLI and Console retained views: retry budgets, exhausted pauses, assessment validation/replay, Session attribution and rechecks.

Evidence

  • Before: the writer lifecycle, terminal-only isolation and Model Score/recheck tests failed at the unimplemented storage interface.
    After: all 13 storage and retained-view tests pass within the full suite, including checkpoint/resume, captured absence, mechanical accounting, partial assessment replay and raw-evidence isolation.
  • cargo check --all-targets — passed.
  • cargo test --test experiments — 78 passed, 1 expected subprocess entry point ignored; includes Stop/resume, fixed Void budgets, Model Grader validation/reuse and rechecks.
  • /usr/bin/time -p scripts/affected-tests main selected cargo nextest run — 2,825 passed, 60 skipped. Test execution: 1,429.495 seconds; total wall time: 1,497.47 seconds, including 1m 06s compilation.

Unaddressed findings

Standards

None. The sole P3 heuristic finding, repeated published-Trial setup, is addressed by published_trial() in commit 4c0fad9f; the reviewer confirmed the fix.

Spec

None. No findings.

Security

None. No introduced findings. Guidance-mode review of the Trial-results storage change found every file write still resolves only against directory-selected entries, Trial keys stay fixed-charset hashes, and terminal-only reads stay independent of pending files. Verified by 7 storage tests, 6 retained-reading tests and 12 Model Grader integration tests, all passing on this branch with no code changes required.

Security review outcome

No unaddressed introduced findings.

Review coverage

Both independent reviewers read all 14 changed files and reviewed the fixture follow-up.
Unread changed files: none under either axis. Generated-output exclusions: none.
Final reports, including each axis's files-read list, are retained at
.thirdshift-review-svcosY/standards.md and .thirdshift-review-svcosY/spec.md.

Merge Danger

Door: two-way

Internal storage ownership change; existing saved data remains compatible.

Blast Radius: Experiments

Built with codex · gpt-6.1-sol · xhigh

@JacobStephens2
JacobStephens2 merged commit 4ed03e7 into main Oct 25, 2026
18 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1288 branch October 25, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deepen Trial results behind one storage interface

1 participant