Cut merge-group CI from ~46 to ~20 min: shard Gradle e2e and test legs, skip test-release in queue, cancel orphaned runs

[Mikola Lysenko (mikolalysenko)]

Why

Merge-group CI takes ~46 min per entry (#1032's run 37777175251: 12:29→13:15Z; #768's and #1050's runs took about the same), and that sets how fast the merge queue lands PRs. Job timings from that run:

job	start (min)	duration (min)
e2e Gradle agent+hosted ×4 (6.9.4 / 7.6.6 / 8.14.3 / 9.8.0)	10	28–35 ← critical path
test (windows-latest)	0	31 (25.7 of it running tests)
test-release	0	30 (23.5 compiling ~240 optimized test binaries)
test (macos-latest)	0	26
yarn-classic / yarn-berry / cargo-vex / cargo-old-toolchains	31 (waiting on test)	1–3

Within a Gradle leg, e2e_redirect_gradle_build alone took 31 min: 43 hosted tests, each a real Gradle build (~2–3 min apiece).

What

1. Shard the hosted Gradle suite. Each Gradle line's single agent+hosted leg becomes three:

   • the agent suites plus gradle_hosted_[345]* (~10 hosted tests)
   • gradle_hosted_[b-p]* (~16)
   • a catch-all: gradle_hosted_ with --skip on exactly the words above (~17), so a newly added test always runs somewhere.

   The new HostedShards tests in scripts/tests/test_ci_gradle_prefixes.py parse the real suite source. For each line they check that every hosted test runs in exactly one leg, and that the catch-all's skip list equals the other legs' words. test_ci_e2e_tiers.py's pinned row table is updated (9 → 17 Gradle rows). Coverage is unchanged: same tests, same Gradle lines, all on ubuntu.

2. Drop needs: [test, coverage] from yarn-classic-matrix, yarn-berry-e2e, cargo-old-toolchains and cargo-vex-matrix (which keeps e2e-build for its binaries). They consume nothing from those jobs, so they were just waiting ~31 min on the Windows test leg. Those jobs used to inherit the draft skip through needs, so 09aded0 gives them their own if: github.event.pull_request.draft != true (my first push missed this).

3. Shard the test legs (macOS + Windows), two per OS, using the new scripts/ci-test-shard.py. The Windows leg spent ~10 min linking ~240 test binaries and ~15 min running them. Shard 1 runs the unit tests, the doctests and a third of the integration targets (from cargo metadata); shard 2 runs the rest. scripts/tests/test_ci_test_shard.py checks that the shards together run exactly the old cargo test --workspace selection, with no target dropped or doubled. A renamed target fails loudly (--test <missing> is a cargo error). macOS shards run only on main, the queue and nightly, as before.

4. test-release skips merge_group. Every PR already runs it on its head, and it still runs on main after each merge. In the queue it re-spent ~30 min (23 of them compiling) per entry. ci-ok counts a skipped job as passing.

5. New merge-queue-janitor.yml. On each merge-group event it cancels queued or running merge-group runs whose gh-readonly-queue/... ref has been deleted, because the queue rebuilt or dropped that entry. Those runs can never merge but kept burning ~200 jobs each, macOS included. Only a definite HTTP 404 on the ref cancels a run; any other lookup error leaves it alone. It is not a required check and never fails. A dry run against the live queue correctly saw the four current entries as live.

Expected: per merge-group entry, about 46 → ~20 min. The Gradle legs come down to ~23 min, the test legs to ~15–18 min, and test-release is gone from the queue path. The cost is more ubuntu/Windows jobs per run (~1–5 min setup each). Doubling the macOS test legs adds ~5 macOS runner-min per main/queue run, which the janitor more than pays back by killing orphans.

Validation

python3 -m unittest discover -s scripts/tests: 264 tests OK. I checked cargo test --workspace --test <cli target> --test <core target> selects across packages and that --doc runs cleanly. Both workflow files parse with no duplicate keys. The YAML parses with 17 Gradle e2e rows and the intended needs.

🤖 Generated with Claude Code

---

Note

Medium Risk
CI-only changes that alter which tests run when (merge queue vs PR) and how Gradle/test filters partition work; mistakes could miss tests or block merges until caught by the new shard/prefix tests.

Overview
Shortens merge-queue CI by parallelizing the slowest legs and dropping redundant work, targeting roughly 46 → ~20 min per queue entry.

test (macOS/Windows) now runs two shards per OS via scripts/ci-test-shard.py instead of one serial cargo test --workspace; shard 1 carries unit tests, doctests, and a lighter share of integration targets. test-release is skipped on merge_group (still on PRs and main) because the queue was re-running the same ~30 min release compile.

Gradle hosted e2e splits from one leg per Gradle line into three parallel legs (named gradle_hosted_* filters plus a catch-all with --skip), with HostedShards / tier tests guarding full coverage.

Job graph: yarn-classic-matrix, yarn-berry-e2e, and cargo-old-toolchains no longer needs: [test, coverage]; cargo-vex-matrix only needs e2e-build. Each gets an explicit draft if where needed.

New merge-queue-janitor.yml cancels in-flight merge_group workflow runs whose gh-readonly-queue/... branch ref is gone (404), so orphaned ~200-job runs stop burning runners.

^{Reviewed by Cursor Bugbot for commit a1f2d34. Configure here.}

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a1f2d34. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Ready for review (burn-down agent).

• Head: a1f2d347d2334dc561a063c7f475db204f7ca4c8
• CI: all check runs green (success/skipped/neutral) on this head; mergeable, no conflicts.
• Bugbot: reviewed this head, no new issues. Its earlier finding (draft PRs running ungated matrix jobs) was fixed in 09aded0 and the thread is resolved.
• Changelog: untouched.

Already approved. Note #1143 is stacked on this branch.

---

Generated by Claude Code