[miguel-heygen]

What changes

Two changes to how the edit accuracy shards are split, both measured on real runs:

1. A pull request runs the pr slice (271 cases) in 15 shards instead of 5. The merge queue keeps its full grid in 20. The gate's and retry job's shard counts move with it.
2. Shards are packed by measured case time instead of round-robin by grid index. Each case's median seconds over 5 merge-queue runs lives in edit-accuracy/timings.json (test:edit-accuracy:timings regenerates it). Shards are packed longest first, the same bin packing plan-regression-shards.mjs uses, and each starts with its slowest cases. An untimed case counts as the slowest.
3. The slow-runner stop judges runner speed. It used to stop a shard when its first 4 cases took a median over 60 s, which assumed grid-order cases. It now stops when they take a median of over 3x their usual seconds (the "3x a healthy runner" it always meant), so the order cases run in no longer matters. Before this, a packed shard starting with its slowest cases would trip on a 1.21x-1.45x runner. Round-robin on main could already trip at 1.33x.

(Replaces #6778, which carried step 2 and was blocked on the slow-runner stop.)

Why

The shards are the critical path of every run that touches Studio.

• Pull requests: over the last 20 green PR runs, Studio: edit accuracy (k/5) took a median 893 s and up to 1243 s. Every other job ends by about 525 s.
• Merge queue: the 20 shards get 672 to 1090 s of case time each under round-robin; packed, it is 891 s each.

Runner speed also varies: every case in a shard runs at about the same multiple of its usual time, 0.72x at p10, 1.43x at p90 and 1.68x at worst over 100 merge-queue shards. Packing can't change that. Simulated with runner multiples drawn from those 100, the expected slowest merge-queue shard drops from 748 s to 686 s, about 60 s per run.

After

Step 1, 15 PR shards (run 40998022015, round-robin split):

	5 shards (before)	15 shards
slowest shard job	893 s median, up to 1243 s (last 20 green PR runs)	412 s (229 to 412 s)
edit accuracy gate finished	1145 s and 1221 s (two recent 5-shard runs)	569 s
shard runner minutes	about 62	80

Steps 2 and 3, packing and the speed-based stop (run 41002343889, 15 shards):

	round-robin (run 40998022015)	packed (run 41002343889)
shard jobs	229 to 412 s	165 to 419 s
runner speed multiples across shards	not measured	0.58x to 1.70x
slow-runner stops / retries	0 / 0	0 / 0

On the PR slice in 15 shards, one run cannot show packing's gain: the runner multiples (0.58x to 1.70x) dominate. The run did show the stop fix working. Under the old 60 s rule, two packed shards started with first-four medians of 57.0 s and 58.8 s, within 3 s of a needless re-run, on runners of about 1.45x. The new rule put them at 1.43x and 1.45x, against a 3x cut-off.

Merge-queue numbers follow from the first queue runs after this lands, compared as a median over several runs, because one run is dominated by runner speed.

No visible change

Only the edit accuracy bench (under packages/studio/tests/), a packages/studio/package.json script, packages/studio/AGENTS.md and ci.yml change. Studio itself is untouched.

Checks

• bunx vitest run packages/studio/tests/e2e/edit-accuracy/grid.test.mjs packages/studio/tests/e2e/edit-accuracy/report.test.mjs: 17/17, three runs in a row.
  • Every case lands in exactly one shard, and shards stay within one case's time of each other: full grid in 20, PR slice in 15. Round-robin fails this with a 418 s spread.
  • Every packed shard, starting with its slowest cases on a 1.68x runner, passes the slow stop, and a 3.36x runner trips it. Put back the 60 s rule and both cases fail.
  • report.test.mjs: the stop compares against usual seconds, so long cases on a normal runner pass.
• scripts/edit-accuracy-retry.test.mjs, check-edit-accuracy-filter.test.mjs and merge-queue-workflows.test.mjs: 7/7. oxlint, oxfmt, the comment ratchet, comment citations and fallow audit pass.

[heygengenesis]

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits · head 8636780d

• blocker: the head no longer matches the PR. The title, the body ("10 shards instead of 5", "+3 runner minutes") and 05a2699 describe 10 shards. 8636780 moved all three counts (ci.yml:1194, :1268, :1324) and the comment at :1182 to 15. Your own table puts 15 at 558 s for +6 runner-minutes, not 707 s for +3. The squash title is what goes on main, so it has to say what it does. Either go back to 10, or update the title, body and rationale for 15. Also fill in ## After, which still says "filled in from that run".

Scope

The three shard counts agree with each other, and the retry plan and gate derive shards from artifacts, so 15 would work mechanically. Locally, edit-accuracy-retry, check-edit-accuracy-filter and merge-queue-workflows pass 7/7. CI on 8636780 is still running.

[github-actions]

Edit accuracy: accurate 271 (base branch 271), smooth 103 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

[heygengenesis]

Verdict: APPROVE — 1 resolved, 0 open, 0 new · 8636780d (same head)

✅ The title and body now describe 15 shards, and ## After is filled in from run 40998022015. All 15 shards and the gate passed, and the gate ended at 569 s. CI, Windows render verification, CodeQL and regression are green on this head.

[heygengenesis]

Verdict: APPROVE — 1 resolved (carried from #6778: slow-runner stop vs longest-first order), 0 open, 0 new · 8636780d → 6f9590ed

✅ The stop now compares runner speed against each case's usual seconds. Locally, grid and report tests pass 17/17, and both 1.68x pin tests fail when I restore the old 60 s rule. The only reader of slow-runner.json is the CI step that checks the file exists, so the new shape is safe. CI and Windows render verification are green on this head.