Skip to content

fix(ci): a release fails its checks when its notes miss a commit merged after the cut - #7601

Merged
miguel-heygen merged 3 commits into
mainfrom
fix/release-notes-complete
Nov 12, 2026
Merged

miguel-heygen merged 3 commits into
mainfrom
fix/release-notes-complete

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

What

A release PR now fails its release check when its notes do not link every commit merged since the last release. The merge queue runs the same check against the main the release will land on, so a commit merged after the notes were drafted can no longer ship unlisted.

Why

The v0.9.38 notes were cut on a main that then took two more merges (#7593 and #7590). Merging would have published both without listing them, and the next draft starts from v0.9.38, so neither would ever have been listed. A reviewer caught it by hand. This makes the check automatic.

No visible change

CI and release tooling only. No Studio, player or CLI behaviour changes.

How

  • scripts/ci/release-only-diff.mjs --verify (the owner of "is this release consistent") now also finds the last chore: release vX.Y.Z commit before the base, lists the commits since, and reports any whose sha the notes do not link. Commits that notes leave out (releases, version bumps, [skip changelog]) never count.
  • The last release is found by its commit subject (chore: release vX.Y.Z, also chore(release): cut vX.Y.Z), never by a line inside another commit's message. When notes leave a commit out on purpose, the failure says to link its sha; it only suggests rebasing and rerunning release:prepare if main moved.
  • That skip rule moved into scripts/changelog-skip.mjs so draft-changelog.ts and this check share one copy.
  • The release-only action now judges the diff in the merge queue too, as a new release_diff output. release_only stays false there, so the queue still runs every suite. The Release checks job also runs when the queue holds a release diff.
  • Search first: no existing completeness check; grep -rn "unlisted\|misses" scripts/ci found none.

Test plan

  • node --import tsx --test scripts/ci/release-only-diff.test.mjs scripts/draft-changelog.test.ts scripts/ci/base-file.test.mjs scripts/merge-queue-workflows.test.mjs: 34 passed, 0 failed, three runs in a row.
  • fails without the fix: --verify fails a release whose notes miss a commit merged since the last release. With the completeness check removed, that test fails and the other 11 in the file pass. A second test (the last release is found by its subject...) goes red alone with the old line-grep anchor put back.
  • oxfmt --check and oxlint on the changed files are clean. tsc -p scripts/tsconfig.json reports the same 3 errors as main (generated files absent), none in the changed files.
  • CI on this PR. A real merge-queue run of a release is the last proof: I did not simulate the queue, only the same check against a temporary git history.
  • Release checks should become a required check so the queue enforces this; a skipped required check passes, so ordinary PRs are unaffected. That setting is a maintainer's.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 101 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 1 important · 1 nit · head 83a43002

I ran the new check against the last 12 releases on main. 8 of them would have failed, and most of those were real late merges: #7577 in v0.9.37, plus misses in v0.9.32, v0.9.33 and v0.9.34. So the check catches the bug it targets. The fails-without-the-fix test is confirmed: with the completeness check stubbed out, test 7 fails and the other 11 pass.

  • important · scripts/ci/release-only-diff.mjs:74: the remedy is wrong when the notes leave a commit out on purpose. v0.9.31 replaced #7502 and #7495 with an unlinked "Hardened release build." line, and this check would fail it. In that case, rebasing and rerunning release:prepare fixes nothing and can overwrite the hand edits. The message should also say "or link each listed sha in the notes".
  • nit · scripts/ci/release-only-diff.mjs:101: --grep matches any line of the message, so a squash body with a line starting chore: release vX.Y.Z moves the anchor forward and silently shrinks the window. It also misses non-canonical release subjects: v0.9.28 anchored past chore(release): cut v0.9.27 and reported 28 commits. Matching subjects the same way as releaseVersionsInSubjects would fix both.
Scope

I read all 6 files. Local release-only-diff.test.mjs: 12/12 pass. CI is green on this head. BLOCKED is the review gate only. Release checks is not a required check in the main ruleset, so the queue will not eject a release until a maintainer makes it required. The PR body already says so.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 2 resolved, 0 open, 0 new · 83a43002 → 12909b8f. The new subject-anchor test fails on the old head and passes here (13/13). Replayed against v0.9.28, it now anchors on chore(release): cut v0.9.27.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 12, 2026
Merged via the queue into main with commit 5d2914f Nov 12, 2026
129 of 130 checks passed
@miguel-heygen
miguel-heygen deleted the fix/release-notes-complete branch November 12, 2026 11:59
@miguel-heygen miguel-heygen mentioned this pull request Nov 12, 2026
3 of 5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant