Repository navigation
Grade Experiment Trials and publish Verdicts - #1131
Merged
Merged
Conversation
This was referenced Oct 23, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #1004
Test seams
Evidence
After: the Experiment integration tests pass with Scores, intervals, correct Inbox grouping and private results.
After:
cargo test --test experiments spec_review_ -- --nocapturepasses all 3 retained regressions, covering agy and OpenCode too.cargo clippy --all-targets -- -D warningspass locally and in CI. The full Rust CI job, site and Console jobs pass onea4dae0f9bcbc0e95e457cba0171b8fc84125666.NEXTEST_TEST_THREADS=2 NEXTEST_STATUS_LEVEL=pass /usr/bin/time -p scripts/affected-tests b8b9ec9passed all 43 site tests. Its redundant local Rust run was stopped after the exact-head full Rust CI job passed, followingdocs/testing.md's CI-reuse rule. The selector took 782.73 seconds including compilation and the interrupted Rust run; this is not a completed full-suite benchmark.Merge Danger
Door: two-way
Blast Radius: Experiments
Private results gain mechanical Scores, suite Verdicts and minutes. Model grading and Trial isolation remain separate tickets.
Unaddressed findings
Standards
None. The P3 Arm-minutes representation finding was addressed with a typed record; the reviewer confirmed the correction.
Spec
None. All 3 P2 findings were reproduced with the reviewers' tests, fixed, and retained as regressions. No decision was deferred and no triage issue was needed.
Review coverage
Both axes read all 16 changed files, including the generated manual's changed hunks. Unread changed files: none. Generated-output exclusions: none.
Fixed point:
issue-977, pinned to its fetched remote tipb8b9ec9c7c42a7d0e47b50ddca85509782cb85d6. Reports, final files-read lists and dispositions are saved locally in.thirdshift-review-b240Hz/standards.mdand.thirdshift-review-b240Hz/spec.md.Built with codex · gpt-6.1-sol · xhigh