Repository navigation
Record plan-limit stops as paused Commands with reset details - #1659
Merged
Merged
Conversation
Quota window names from a Trial session file are copied into the pause message and Command record. Keep only short harness tokens so a forged name cannot change what the operator sees.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Plan-limit stops now keep their own Command outcome and the facts needed to resume.
Finished Trials and Scores remain saved. Ordinary failures and older failed records keep their existing outcomes. Pauses neither send failure email nor count as failed outcomes. Automatic resume remains out of scope (#1653).
Closes #1619
Evidence
Before:
cargo test --test experiments plan_limit_warnings_pause_both_workers_and_rerunning_resumes_void_trials -- --exact --nocapturefailed: the Experiment state waspaused, but the Command outcome wasfailed.After: the same regression passes, including both Harnesses, null reset fallback and resuming Void Trials.
Before:
cargo test --test experiments model_graders::plan_limit_in_grading_keeps_trials_and_shows_a_resumable_paused_command -- --exactfound missing limit facts after completed Trials reached the Grader.After: the regression covers Claude's 86% seven-day stop, RFC 3339 reset,
show --json, Dismiss/Undo, and removal from Needs you while resumed grading is still running. Finished Trials are not repeated.Scoring regression covers Claude warnings/rejections and Codex native reports, retaining completed Scores on resume. Decoder checks preserve each stopping window's own reset and explicit unknown fields.
node --test --test-name-pattern='Paused rows and Command details' console/tests/inbox.test.cjspasses: both locations copy the exact saved command to the clipboard. This exposed and fixed compact-row visibility and event delegation.CI exposed copy feedback disappearing during a live Inbox refresh. A forced-refresh regression reproduced it; the existing
data-copy-statuspreservation keeps feedback visible.cargo test --test console urls_in_inbox_and_peek_text_are_links_and_other_text_stays_escaped -- --exactalso passes after limiting the new cause markup to paused rows.Spec review's complete standalone parser reproduction failed before its correction (
Noneinstead of the known matching reset) and passed unchanged afterward. Its case is retained insrc/harness/interpretation_tests.rs.cargo clippy --all-targets -- -D warningspassed locally and in CI.scripts/affected-tests mainpassed its local site checks (91.35 s) and all 126 Console browser checks (139.83 s). Its duplicate local Rust run was stopped after successful CI on the same final head, followingdocs/testing.md's CI reuse policy.Full Linux CI on
2d704eee0019ca5a66edde53700bfa6f6b5f2e14: 3,610 Rust tests passed, 75 skipped, across 79 binaries. Nextest test runtime: 1,145.120 s, excluding compilation. The macOS gate, site and Console CI jobs also passed.Test seams
show,status, and Dismissal.Review coverage
Standards: 0 findings. Spec: 1 finding, fixed and verified. Both axes read all 41 changed paths through
42cc24f9, includingsite/docs/index.html, and reviewed the Spec correction. No generated-output exclusions. The final CI repair hunks insrc/console/row.htmlandconsole/tests/inbox.test.cjsat2d704eeewere not reread by either axis after their one follow-up; both earlier versions were read. Final reports and counts are saved in.thirdshift-review-JxoMUH/for the Session trace.Unaddressed findings
Standards
None.
Spec
None. The matching-window reset finding was reproduced with its exact command, fixed, and retained as a regression.
Security
None.
Security review outcome
No unaddressed introduced findings.
Merge Danger
Door: two-way
Additive Command fields; older records remain readable without migration.
Blast Radius: outcomes
Command readers now distinguish plan-limit pauses across the Inbox, Console and status. Paused Scoring returns a nonzero exit code instead of reporting a completed sample; rerunning continues pending work.
Built with codex codex-cli 0.164.0 · gpt-6.1-sol · xhigh