Skip to content

ci: edit accuracy shards start once the CLI is built, not after all of Build - #6777

Merged
miguel-heygen merged 1 commit into
mainfrom
ci/edit-accuracy-own-cli-build
Oct 30, 2026
Merged

miguel-heygen merged 1 commit into
mainfrom
ci/edit-accuracy-own-cli-build

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

What changes

The edit accuracy shards only need packages/cli/dist, but they waited for the whole Build job, which goes on to pack every package (verify:packed-manifests) and check the CLI's lazy dependencies after the CLI already exists. A new job, Studio: edit accuracy CLI build, runs the same install and bun run build and uploads the CLI; the shards and their retry job need it instead of Build.

Build keeps every step and stays the required check. It no longer runs just because the edit accuracy filter matched, since that clause (from #5566) only existed to produce the artifact.

If the new job fails, the shards are skipped and Studio: edit accuracy gate fails because the shards did not run (#6069), so a broken CLI build still blocks the merge.

Why

The shards are the merge queue's critical path: Detect changes -> Build -> the slowest of 20 shards -> the gate.

Before, over the last 20 green merge-queue CI runs: Build took a median 266 s, of which verify:packed-manifests took a median 98 s and check:cli-lazy-deps 4 s, all after the CLI was built. In run 40985995480 the CLI build finished 114 s into Build; the shards started 202 s in.

After

This PR's own run (40989910509, pull request, 5 shards): Studio: edit accuracy CLI build ended 195 s into the run and the shards started at 197-198 s; Build ended at 285 s. Before this change the shards could only start after Build, so they now start about 87 s sooner, and the run's critical path is shorter by the same amount. Merge-queue numbers follow from the first queue runs after it lands.

Checks

  • node --import tsx --test scripts/check-edit-accuracy-filter.test.mjs scripts/edit-accuracy-retry.test.mjs scripts/merge-queue-workflows.test.mjs scripts/ci-cache-workflows.test.mjs: 9/9 pass. The filter test now reads the new job's steps for what the bench is built from.

Size

Under 100 lines on purpose: one workflow dependency change that takes 87 s off every CI run; nothing else open touches ci.yml to carry it.

@github-actions

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 167 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 0 important · 0 nits · head 0c89941a

Scope

Read the ci.yml diff in context. The only thing that consumed the edit-accuracy-cli artifact was the shards and their retry, and both now need the new job. Dropping edit_accuracy from Build's if only skips Build on studio-only diffs, and a skipped Build reads as passing. It was only there to produce the artifact. Fail-safe holds: if the CLI build fails, the shards are skipped and ratchet.mjs gate --shards returns "0 of N shards reported results", so the required gate fails. The 4 workflow test files pass locally (9/9). Both CI runs on this head are green.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit c922591 Oct 30, 2026
166 checks passed
@miguel-heygen
miguel-heygen deleted the ci/edit-accuracy-own-cli-build branch October 30, 2026 03:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant