Repository navigation
Deepen Experiment assessment behind a read-only interface - #1308
Merged
Merged
Conversation
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Experiment storage obtains the provisional Verdict, suite assessments and pending rechecks through one read-only grading assessment.
Pending record serialization, human history publication, legacy upgrades and CLI/Console behavior stay compatible. Navigation now identifies grading as the assessment owner.
Closes #1307
Evidence
After: storage and completed-Experiment upgrades use one shared-input assessment; five focused tests cover complete observations, whole-Experiment decisiveness across suites, separate flips, human overrides, Void pairs, validation errors, repeatability and unchanged inputs.
cargo check;cargo test --bin thirdshift evals::experiment::grading::tests(5);cargo test --test experiments rechecks::(7);cargo test --bin thirdshift verdict::tests(10)./usr/bin/time -p scripts/affected-tests mainpassed, selectingcargo nextest run: 2,935 passed, 62 skipped, no failures. Wall time 1,499.92 seconds, including compilation (55.97 seconds); nextest execution was 1,442.051 seconds. Timing depends on this host and build cache. Full output is saved beside the Session trace in.thirdshift-review-jzhTLU/validation.log.674e6e5ba27e1d3fc83d9c1ed64b94e457189314, including Linux tests, site, Console, the macOS gate and release artifact checks.Test seams
grading::assess(&[Task], &[TrialResult], &Bar): complete Verdict and pending records, repeatability, unchanged inputs, paired validation, effective human Scores, Void exclusion and whole-Experiment decisive flips.Review coverage
Independent Standards and Spec reviews used
git diff main...HEAD, withmainpinned atd414b0f108fe3702e47eaaa8fc469000396ca5a1. Both axes reported zero findings and read every changed file:docs/code-navigation.mdsrc/evals/experiment.rssrc/evals/experiment/grading.rssrc/evals/experiment/grading/tests.rssrc/evals/experiment/rechecks.rssrc/evals/experiment/store.rsUnread changed files: none on either axis. Generated-output exclusions: none. Full reports and each axis's final supporting files-read list are saved beside the Session trace in
.thirdshift-review-jzhTLU/standards.mdand.thirdshift-review-jzhTLU/spec.md.Unaddressed findings
git diff d414b0f108fe3702e47eaaa8fc469000396ca5a1...HEADplus supporting code found no introduced finding: the change is a pure in-process refactor (mutable flip-and-restore counterfactuals become projected single-scalar flips over read-only inputs) with no new trust boundary, principal, sink, file path, network call, or serialized-format change; aggregation, Void-exclusion, human-overlay and recheck-gating rules are preserved. No repository SECURITY.md or threat model exists. Validated withcargo test --bin thirdshift evals::experiment::grading::tests(5 passed),cargo test --test experiments rechecks::(7 passed) andcargo test --bin thirdshift verdict::tests(10 passed). No proof-of-concept failed, so no fix was made; pre-existing findings, if any, are recorded only in the private security-review record.Security review outcome
No unaddressed introduced findings.
Merge Danger
Door: two-way
Reversible implementation ownership change with no saved-format migration.
Blast Radius: Experiments
Verdict assessment and pending recheck selection; recording and views retain their existing interfaces.
Built with codex · gpt-6.1-sol · xhigh