Skip to content

Wait for release Experiments before publishing - #1235

Merged
JacobStephens2 merged 4 commits into
issue-977from
issue-1022
Oct 24, 2026
Merged

JacobStephens2 merged 4 commits into
issue-977from
issue-1022

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Summary

Releases wait for an Experiment against the last release tag on touched Eval suites.

release content commit (pinned across interruption)
  → Experiment --release → touched suites vs last tag
      pass                → continue
      pending rechecks    → list Scores and poll the same Experiment
      fail / inconclusive → show intervals and ask to release anyway
      no suite / old tag without Trial flag → note and continue

A pushed bump branch is checked again before opening its PR. Update the manual and hand-written site pages. Wait for fonts before browser layout assertions to avoid measuring a fallback typeface.

Closes #1022

Evidence

  • Before: release mode was rejected as an unknown flag. The Spec review's exact interrupted_release_before_push_keeps_its_experiment_when_main_moves regression failed with two different Candidates.
    After: the release CLI tests pass, including real SIGINT/resume. The exact review regression passes with the original Candidate retained.
  • Fake-cargo release tests cover pass, fail/inconclusive overrides, pending rechecks, resume, and skip notes. The compact-help regression and both focused 320px layout checks pass. Clippy with warnings denied and all 14 Python tooling tests pass.
  • /usr/bin/time -p scripts/affected-tests 42bad19bb424fd31111add2ed36eb05034a5eb74 passed: 2,563 Rust tests (58 skipped), 44 browser tests, and the selected release-generator Python check. Total wall time: 1,388.57 seconds, including 11.84 seconds of Rust compilation; Nextest execution: 1,318.191 seconds.
  • CI passed on the exact pushed head cb172f4c.

Test seams

  • Experiment CLI: release invocation and public saved Experiment listing/views, using real Git repositories and cached Arm binaries.
  • Release script: end-to-end release fixture with fake cargo at the external command seam.
  • Existing browser layout assertions against the rendered pages.

Review

Independent Standards and Spec reviews against issue-977 at 42bad19b: both Standards findings and the Spec finding are addressed. The Spec finding's supplied test was run verbatim, failed before the fix, and passes afterward.

Both axes read all 16 implementation/review-fix files, including generated site/docs/index.html; no generated output was excluded. A subsequent fixture change, site/tests/pages.test.cjs, was left unread by both axes and is disclosed in both saved reports. It waits for fonts before the existing layout assertions; the focused tests pass.

Unaddressed findings

  • Standards: None.
  • Spec: None.
  • Security: None (guidance-mode review of the release Experiment change against 42bad19b; no introduced finding confirmed; the 6 fake-cargo release-gate tests pass).

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

Blast Radius: Releases

Adds a local release-candidate Git ref until the branch push. Fail or inconclusive Verdicts require explicit permission; EOF stops before publishing more release artifacts.

Built with codex · gpt-6.1-sol · xhigh

@JacobStephens2
JacobStephens2 merged commit 298b6b8 into issue-977 Oct 24, 2026
18 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1022 branch October 24, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant