Repository navigation
ci: caches survive lockfile changes; no re-cloning; blobless OpenSecret checkout - #1342
Merged
Merged
Conversation
Every ci.yml lane cache was keyed by sha256(flake.lock + Cargo.lock) in rust-cache's shared key with add-rust-environment-hash-key off, so the restore key equalled the cache key and any Agent Cargo.lock edit made every Agent Rust lane fully cold. That hit 25% of PRs, two thirds of them version bumps: PR #1338 (a version bump) took 10.6 min to ci-ok instead of about 4.5, and its master push 18.6 min instead of about 10.5. - The shared key now holds only the toolchain pins (flake.lock, and rust-toolchain.toml where the workspace has one), which must match. - rust-cache's environment hash (rustc versions, CARGO*/RUST* variables) ends the restore key, and its lockfile hash (third-party Cargo.lock entries, member manifests with versions zeroed) ends only the cache key (config.ts at e18b4977). A version bump restores its exact cache; any other lockfile change restores the newest cache of the same lane and toolchain. Of the last 30 Agent Cargo.lock commits, 13 keep the exact key and 17 fall back by prefix; today all 30 start cold. - The environment hash also covers the runner image's preinstalled rustup toolchains, so an image with a new Rust costs one cold run per lane until master saves again. - Prefix ci-v2, so no lane cache can fall into the ci-v1 release families; a C4 contract test pins that and the rust-cache flag. - agent-windows-updater also runs on master pushes (free windows-2025), so a master lockfile change saves its cache instead of leaving PRs cold until the nightly run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agent-supply-chain and agent-provider had no cache, so every run cloned the zed and trycua/cua git dependencies from GitHub only to run `cargo metadata`: 0.3-11 min of fetching against 1.1-2.4 min of real work (job 122541527929 on PR #1336: 5.4 min on cua, 4.7 on zed). Both lanes ran at p90 10.7 min and are selected on about 30% of PRs. They now share one rust-cache entry, agent-cargo-home, that holds only the Cargo home (registry index and crates, git databases and checkouts): a new lane field, cache_targets=False, drops the target directory. PRs restore it; master pushes and the nightly run save it into the lanes' home store, like every lane cache. These lanes run on the PRs that edit Cargo.lock, where the prefix fallback from the previous commit restores master's last graph, so Cargo fetches only what changed. Expected p90: about 2.5 min. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OpenSecret jobs need full history (Nix requires revCount for the recursively fetched submodule), and on Blacksmith that clone has been slow since 10-27: checkout p50/p90/max os-eif 231/557/601 s, os-rust 76/475/527 s, os-sdk-e2e 21/333/427 s (job 122029252679 spent 8 min in "Fetching the repository"). The plan job already clones the same history blobless in 5-17 s. os-rust, os-flake-check and deps-opensecret (through a new `filter` matrix field from the planner), os-sdk-e2e and the os-eif PR candidate now check out with filter: blob:none, keeping fetch-depth 0 and recursive submodules: every commit and tree, but file contents only for the checked-out tree (39 MB instead of 346 MB). The submodule is still fetched in full. opensecret-eif-release.yml and the PCR checks are unchanged. Nix sees byte-identical source. Emulating actions/checkout on 7cdc65c with Determinate Nix 3.24.0, a full clone (20 s, 346 MB) and a blobless clone (7 s, 39 MB) give the same `git ls-files -s --recurse-submodules` digest, submodule commit and working tree, the same flake source (revCount 4091, narHash sha256-2mRvfa57+T2G0pQzpclNWod339i++bnhkfqFXHHy0D8=, one store path) and the same derivations for eif-dev, eif-prod, the ci dev shell and every checks.x86_64-linux entry. Git objects are content addressed, so a missing blob could fail a build but never change its source. Expected: os-rust p90 about 9.3 to 4.4 min, os-sdk-e2e p90 9.8 to 5.7, os-eif up to 10 min less checkout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rust-cache's save keeps every build of a crate still in the dependency graph, so a master run that restored by prefix would carry the old graph's builds into each new cache. A master run that will save a cache without an exact hit now deletes the lane's target directory first; pull requests keep the prefix fallback and version bumps stay exact hits. The lane cache contract also checks that step, and checks key overlap in both directions, including other workflows' rust-cache steps. The cache docs name the Windows lanes' toolchain pin correctly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AnthonyRonning
force-pushed
the
ci/pr-speed-caches
branch
from
October 28, 2026 05:36
07f73fa to
159fc85
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Faster PRs, at no cost. Estimates come from a replay of the last 100 merged PRs with measured lane times:
agent-supply-chain/agent-providerp90os-rust/os-sdk-e2ep90Cargo.lockchange.ci-v1-agent-release-*) are untouched.agent-windows-updaternow also refreshes its cache on master pushes, on the free runner.ls-filesdigest, identicaleif-dev/eif-prodderivations. The PCRs therefore cannot change.Known trade-off: the cache restore key includes the runner image's Rust version, so a runner image update runs each lane cold once, until master saves again.
Expect after merge: this PR edits
ci.yml, so its own run and the merge push start every lane cold, once.Local checks pass:
plan.py lint, the scripts/ci suite (514), the release gates and actionlint. The full rationale and evidence are in the commit messages andimpl-notes.md.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.