Skip to content

Research faster macOS CI without dropping macOS support - #902

Merged
JacobStephens2 merged 1 commit into
mainfrom
macos-ci-research
Oct 20, 2026
Merged

JacobStephens2 merged 1 commit into
mainfrom
macos-ci-research

Conversation

@JacobStephens2

Copy link
Copy Markdown
Owner

Summary

Adds docs/research/macos-ci-speed.md: why macOS CI queues, and the options for keeping macOS without the wait.

GitHub personal-account cap: 5 concurrent macOS jobs
  CI run  -> 4 shards x ~18 min
  Release -> 1 aarch64-apple-darwin build
measured 47.5 h: 3.8 of 5 slots busy on average
  < ~3 slots busy  -> median queue 9 s
  4.2-5.1 busy     -> median queue 10-20 min, worst 85 min

macOS runner time by trigger: PRs into main 47%, PRs into Spec branches 29%, pushes to main 23%.

Recommendation, cheapest first:

  1. Run the full macOS suite on main and nightly, with a one-job macOS gate on PRs into main #893 - skip macOS on PRs into Spec branches.
  2. Full 4-shard suite on pushes to main and nightly; a one-job macOS gate on PRs into main, with a test:macos label to opt into the full suite (uv's pattern).
  3. Probe XProtect and per-exec cost under Find what still makes the macOS test shards slow after #857 #894.
  4. Only if main still queues, 2 Cirrus Runners at $150/month each.

Ruled out: GitHub larger runners and merge queues (organization accounts only), a Linux cargo check --target aarch64-apple-darwin gate (aws-lc-sys needs a C cross toolchain), per-minute vendors (~$10-13k/month at today's volume) and self-hosted Macs on a public repo.

Evidence

  • Queue and execution times come from the jobs API over 160 CI and 112 Release runs, 753 macOS jobs, 2026-10-18 17:00 to 2026-10-20 17:00 UTC.
  • Every external claim links its primary source: GitHub docs, actions/runner-images at 68bd77a, nextest docs, vendor pricing pages, and the workflow files of ripgrep, cargo, uv, jj, nextest and starship at pinned SHAs. Claims no primary page confirmed are marked [unverified].

Merge Danger

Door: two-way

Documentation only; no workflow or code changes.

Blast Radius: docs

macOS runs wait in queue because thirdshift's own CI fills GitHub's
5-job macOS cap for personal accounts: 3.8 of 5 slots busy on average
over 47.5 h, with a median 33 min to the last macOS shard against 11 min
on Linux. PRs into Spec branches take 29% of that time, PRs into main
47%, pushes to main 23%.

Recommends #893, then the full suite on pushes to main and nightly with
a one-job macOS gate on PRs into main, before any paid runners.
@JacobStephens2
JacobStephens2 merged commit 9235399 into main Oct 20, 2026
6 checks passed
@JacobStephens2
JacobStephens2 deleted the macos-ci-research branch October 20, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant