Skip to content

ci: caches survive lockfile changes; no re-cloning; blobless OpenSecret checkout - #1342

Merged
AnthonyRonning merged 4 commits into
masterfrom
ci/pr-speed-caches
Oct 28, 2026
Merged

AnthonyRonning merged 4 commits into
masterfrom
ci/pr-speed-caches

Conversation

@AnthonyRonning

@AnthonyRonning AnthonyRonning commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Faster PRs, at no cost. Estimates come from a replay of the last 100 merged PRs with measured lane times:

Before After
Agent version-bump PRs (17 of 100; #1338 took 10.6 min) ~10 min ~4.5–5 min
Agent dependency-change PRs 9–15 min ~5.5–7 min
agent-supply-chain / agent-provider p90 ~11 min ~2.4 min
os-rust / os-sdk-e2e p90 9.3 / 9.8 min ~4.4 / ~5.7 min
PRs over 10 min 36 / 100 ~9 / 100 (the rest wait on the EIF build; #1341 moves it to 16 vCPU)
  1. Lane caches survive a Cargo.lock change.
    • A version bump keeps its exact cache. Other lockfile changes restore the newest cache with the same toolchain.
    • Pull requests still never save a cache.
    • Before saving after a prefix restore, master clears the lane's target directory, so stale builds don't pile up.
    • The release caches (ci-v1-agent-release-*) are untouched.
    • agent-windows-updater now also refreshes its cache on master pushes, on the free runner.
  2. The dependency lanes share a Cargo home cache. They stop cloning zed and cua from GitHub on every run, which took 0.3–11 min.
  3. OpenSecret lanes use a blobless full-history checkout. It downloads 39 MB instead of 346.
    • I verified with a clean, uncached check that Nix sees the same source: same narHash, same ls-files digest, identical eif-dev/eif-prod derivations. The PCRs therefore cannot change.
    • EIF and PCR workflows are untouched.

Known trade-off: the cache restore key includes the runner image's Rust version, so a runner image update runs each lane cold once, until master saves again.

Expect after merge: this PR edits ci.yml, so its own run and the merge push start every lane cold, once.

Local checks pass: plan.py lint, the scripts/ci suite (514), the release gates and actionlint. The full rationale and evidence are in the commit messages and impl-notes.md.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

AnthonyRonning and others added 4 commits October 28, 2026 05:32
Every ci.yml lane cache was keyed by sha256(flake.lock + Cargo.lock) in
rust-cache's shared key with add-rust-environment-hash-key off, so the
restore key equalled the cache key and any Agent Cargo.lock edit made
every Agent Rust lane fully cold. That hit 25% of PRs, two thirds of them
version bumps: PR #1338 (a version bump) took 10.6 min to ci-ok instead
of about 4.5, and its master push 18.6 min instead of about 10.5.

- The shared key now holds only the toolchain pins (flake.lock, and
  rust-toolchain.toml where the workspace has one), which must match.
- rust-cache's environment hash (rustc versions, CARGO*/RUST* variables)
  ends the restore key, and its lockfile hash (third-party Cargo.lock
  entries, member manifests with versions zeroed) ends only the cache key
  (config.ts at e18b4977). A version bump restores its exact cache; any
  other lockfile change restores the newest cache of the same lane and
  toolchain. Of the last 30 Agent Cargo.lock commits, 13 keep the exact
  key and 17 fall back by prefix; today all 30 start cold.
- The environment hash also covers the runner image's preinstalled
  rustup toolchains, so an image with a new Rust costs one cold run per
  lane until master saves again.
- Prefix ci-v2, so no lane cache can fall into the ci-v1 release
  families; a C4 contract test pins that and the rust-cache flag.
- agent-windows-updater also runs on master pushes (free windows-2025),
  so a master lockfile change saves its cache instead of leaving PRs
  cold until the nightly run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agent-supply-chain and agent-provider had no cache, so every run cloned
the zed and trycua/cua git dependencies from GitHub only to run
`cargo metadata`: 0.3-11 min of fetching against 1.1-2.4 min of real
work (job 122541527929 on PR #1336: 5.4 min on cua, 4.7 on zed). Both
lanes ran at p90 10.7 min and are selected on about 30% of PRs.

They now share one rust-cache entry, agent-cargo-home, that holds only
the Cargo home (registry index and crates, git databases and checkouts):
a new lane field, cache_targets=False, drops the target directory. PRs
restore it; master pushes and the nightly run save it into the lanes'
home store, like every lane cache. These lanes run on the PRs that edit
Cargo.lock, where the prefix fallback from the previous commit restores
master's last graph, so Cargo fetches only what changed. Expected p90:
about 2.5 min.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OpenSecret jobs need full history (Nix requires revCount for the
recursively fetched submodule), and on Blacksmith that clone has been
slow since 10-27: checkout p50/p90/max os-eif 231/557/601 s, os-rust
76/475/527 s, os-sdk-e2e 21/333/427 s (job 122029252679 spent 8 min in
"Fetching the repository"). The plan job already clones the same history
blobless in 5-17 s.

os-rust, os-flake-check and deps-opensecret (through a new `filter`
matrix field from the planner), os-sdk-e2e and the os-eif PR candidate
now check out with filter: blob:none, keeping fetch-depth 0 and recursive
submodules: every commit and tree, but file contents only for the
checked-out tree (39 MB instead of 346 MB). The submodule is still
fetched in full. opensecret-eif-release.yml and the PCR checks are
unchanged.

Nix sees byte-identical source. Emulating actions/checkout on 7cdc65c
with Determinate Nix 3.24.0, a full clone (20 s, 346 MB) and a blobless
clone (7 s, 39 MB) give the same `git ls-files -s --recurse-submodules`
digest, submodule commit and working tree, the same flake source
(revCount 4091, narHash sha256-2mRvfa57+T2G0pQzpclNWod339i++bnhkfqFXHHy0D8=,
one store path) and the same derivations for eif-dev, eif-prod, the ci
dev shell and every checks.x86_64-linux entry. Git objects are content
addressed, so a missing blob could fail a build but never change its
source.

Expected: os-rust p90 about 9.3 to 4.4 min, os-sdk-e2e p90 9.8 to 5.7,
os-eif up to 10 min less checkout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rust-cache's save keeps every build of a crate still in the dependency
graph, so a master run that restored by prefix would carry the old graph's
builds into each new cache. A master run that will save a cache without an
exact hit now deletes the lane's target directory first; pull requests keep
the prefix fallback and version bumps stay exact hits.

The lane cache contract also checks that step, and checks key overlap in
both directions, including other workflows' rust-cache steps. The cache docs
name the Windows lanes' toolchain pin correctly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AnthonyRonning AnthonyRonning added the ci:eif Run the OpenSecret EIF PR candidate build label Oct 28, 2026
@AnthonyRonning
AnthonyRonning merged commit b6b246a into master Oct 28, 2026
72 checks passed
@AnthonyRonning
AnthonyRonning deleted the ci/pr-speed-caches branch October 28, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:eif Run the OpenSecret EIF PR candidate build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant