Skip to content

Fix gem vendored unwind missing stale cache (#1594) - #1667

Merged
Mikola Lysenko (mikolalysenko) merged 5 commits into
mainfrom
agent/fix-gem-vendored-unwind-stale-cache
Oct 22, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 5 commits into
mainfrom
agent/fix-gem-vendored-unwind-stale-cache

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #1594

Summary

Say a project ran bundle cache while a hosted gem patch was live, and later took the patch over as vendored. When the vendored gem is unwound (remove, rollback, vendor --revert), the command now warns upstream_gem_stale_cache and names the patched vendor/cache/<name>-<version>.gem, as the hosted unwind already does (#1260). Before this change it exited 0 with no warning. Fresh frozen installs then kept installing the patched bytes, or failed on CHECKSUMS, while the lock said upstream.

Root cause

The Bundler cache-dir check (stale_cache_warning in patch/redirect/upstream/gem.rs) ran only inside the hosted restore. The vendored gem revert (vendor/gem.rs revert_gem_opts) writes the same upstream GEM entry back, but nothing checked the cache after it. The takeover's advisory was the only notice, and it said installs fail now. That isn't true while the vendored path source shadows the archive.

Changes

  • core: stale_cache_warning now takes plain (purl, name, version, lock, client). The new upstream::gem_stale_cache_after_unwind(root, purl, offline) runs the same check after a vendored unwind. It judges the archive against the restored lock's CHECKSUMS sha, or else the rubygems.org compact index (--offline names the archive as unchecked). A gem locked from another remote with no CHECKSUMS sha isn't judged: it can't be checked against rubygems.org, and a committed cache of private gems is normal.
  • cli: VendoredBackend::revert is the shared path behind rollback, remove, vendor --revert and vendor's manifest reconcile. It now adds the warning for a wet gem unwind, next to the existing PyPI reinstall advisory. scan --prune's manifest-dropped leg does the same and forwards it to gc.warnings[]. A dry run skips the check, because the lock on disk is still vendored and can't judge the archive.
  • takeover note: the upstream_gem_stale_cache advisory a takeover raises now adds that bundler installs the vendored copy while the gem stays vendored, and that the unvendoring command names the archive again.
  • CLI_CONTRACT.md's upstream_gem_stale_cache row lists the vendored channels. Each one uses its command's existing convention for vendored warnings: warnings[] for rollback, advisory events for remove, vendor --revert and vendor, and gc.warnings[] for prune.

No wrapper (npm/, pypi/, gem/) changes: they only dispatch the binary.

Test evidence

Issue Test Without fix With fix
#1594 remove / rollback / vendor --revert after a hosted → vendored takeover e2e_redirect_gem_build::gem_vendored_unwind_names_a_patched_archive_in_vendor_cache (real gem build / bundle install / bundle cache, wiremock API; get <uuid> --mode vendored takeover, then each unwind) fails: remove exit 0, events only removed/vendor_reverted, no upstream_gem_stale_cache passes for all three commands; also checks the takeover note's new wording
shared check upstream::gem::tests::vendored_unwind_* (5 unit tests: patched archive named; upstream or missing archive quiet; offline without CHECKSUMS named as unchecked; private remote without CHECKSUMS quiet, but judged when CHECKSUMS exists; a FIFO lock does not block) n/a (new fn) pass

Commands run locally (Linux, Ruby 3.3.6, Bundler 4.0.18):

  • cargo fmt --all -- --check: clean
  • cargo clippy --workspace --all-features -- -D warnings: clean
  • cargo test -p socket-patch-cli --all-features --test e2e_redirect_gem_build -- --ignored: 31 passed
  • cargo test -p socket-patch-cli --all-features --test e2e_vendor_gem_build -- --ignored: 8 passed
  • cargo test --workspace --all-features: 14,674 passed, 10 failed, none in code this PR touches. Nine are permission-failure tests (read-only files or dirs) that can't fail because this container runs them as root (uid 0): in_process_redirect write-failure ×2 plus vlt::scan_redirect_vlt_heal_invalidation_failure_warns, repair ×2, core copy_tree / vlt_heal / pypi_poetry / pypi_requirements. The tenth, e2e_bun_lockb::binary_shared_bundled_record_hosted_pin_is_managed, needs the live patch API, which this sandbox can't reach. CI runs as non-root with network

Review notes (/code-review high)

Fixed: the rubygems.org fallback for private-remote gems, the dry-run lock read, the duplicated lock list (now BUNDLER_LOCKS), the per-entry full VendorEntry clone, and the takeover note and contract missing the reconcile channel.
Not changed, and why:

  • Emit the warning from revert_gem_opts instead. The vendored revert backends are network-free and RevertOpts carries no offline. Threading it through every backend's destructuring is a wider change. The check sits in the shared VendoredBackend::revert path, exactly where the PyPI reinstall advisory already lives. run_vendor_gc leg (b) is left out on purpose: the gem has left the lock there, so a cached archive no longer installs.
  • One HTTP client and cache-dir lookup per gem. An unwind touches a few gems at most, and the compact-index answer is cached per client. Not worth new plumbing.
  • Dry-run takeover note on a preview that later fails. The note is conditional ("while the gem stays vendored"), and the failed event sits next to it in the same envelope.

Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
A project that ran `bundle cache` while a hosted gem patch was live
keeps the patched archive in vendor/cache. After a hosted to vendored
takeover, `remove`, `rollback` and `vendor --revert` put the gem
back on its upstream lock entry but never named that archive, so
fresh installs kept the patched bytes (or failed on CHECKSUMS) while
the command reported success.

The vendored unwind now runs the same Bundler cache check as the
hosted unwind and warns upstream_gem_stale_cache, naming the file.

Fixes #1594

Assisted-by: Claude Code:claude-opus-5-5
The takeover's upstream_gem_stale_cache notice said installs fail
now, but the takeover then wires the vendored copy, which bundler
installs instead. It now says the archive only matters once the gem
is unwired, and that the unwind names it again.

`scan --prune` reverting a vendored gem whose patch left the
manifest runs the same cache check and reports it in gc.warnings[].
Unit tests cover the shared check, and CLI_CONTRACT.md lists where
each vendored unwind reports the warning.

Refs #1594

Assisted-by: Claude Code:claude-opus-5-5
Without a CHECKSUMS sha, the vendored unwind's cache check fell back
to rubygems.org for every gem. A privately sourced gem, which teams
often commit to vendor/cache on purpose, would then be flagged on
every unwind. The fallback now runs only for gems locked from
rubygems.org (or SOCKET_RUBYGEMS_URL); other remotes stay quiet
unless CHECKSUMS can judge them.

A dry run skips the check: it reads the still-vendored lock, which
can't judge the archive. The takeover note now names the unvendoring
command generically, and the contract lists the vendor reconcile.

Refs #1594

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 22, 2026 13:46
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread crates/socket-patch-core/src/patch/redirect/upstream/gem.rs
The vendored unwind's stale-cache check opened Gemfile.lock and
gems.locked with a bare read, so a FIFO squatting either path would
hang the command. It now uses the same regular-file guard as the
hosted restore and skips anything that is not a regular file.

Refs #1594

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 3168aa9. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 22, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review at head 3168aa912d26c3adfb799695e4422d51cb2dbd0c.

  • CI: 35/35 green (19 skipped) on the current head; mergeable, no conflicts.
  • Bugbot: reviewed 3168aa9, no new issues; the one earlier thread (gem.rs lock read) is resolved by 3168aa9's switch to the regular-file guard.
  • For the reviewer: the new upstream_gem_stale_cache warning on vendored unwinds (VendoredBackend::revert, scan --prune) and the reworded takeover note in CLI_CONTRACT.md.
  • Slack announcement not sent this run (Slack send unavailable to the burn-down agent); a later run will announce it.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does: After a hosted → vendored gem takeover, a bundle cache taken while the hosted pin was live leaves the patched .gem in vendor/cache. Unwinding the vendored gem (remove, rollback, vendor --revert, scan --prune, vendor reconcile) used to exit 0 silently. It now raises the same upstream_gem_stale_cache warning the hosted unwind already raises (#1260), naming the archive. Fixes #1594.

Risk: low. The change only adds a warning; nothing is deleted or rewritten. The check returns early unless an archive for the gem is actually in Bundler's cache dir, so the rubygems.org lookup only happens in that case. It is skipped on dry runs and for private-remote gems that have no CHECKSUMS sha.

Look here:

  • gem.rs:1395 unwound_stale_cache:`` reads the lock through the regular-file guard and skips unjudgeable private gems (rubygems_locked, L1421)
  • vendored_backend/mod.rs:164:```` the shared revert path, wet runs only
  • vendor.rs:4659: scan --prune forwards it to gc.warnings[]
  • vendor.rs:648 takeover_detail: reworded takeover note
  • CLI_CONTRACT.md:1436: contract row

Verified:

  • Read the full diff (7 files, +508/−14). The vendored gem backend only works at the project root, so checking cwd's lock is the right scope.
  • A gem missing from a restored lock gets no warning.
  • stale_cache_warning gets the same refactor in the hosted path, with no change in behavior.
  • No debug code or scope creep, and CHANGELOG.md is untouched. The branch is level with main.
  • Ran cargo test -p socket-patch-core --lib vendored_unwind locally: 5/5 pass. The e2e test covers all three unwind commands with real bundler.
  • CI: ci-ok and clippy are green on 3168aa9. Bugbot is clean on 3168aa9, and its one earlier thread is resolved.

Changes I made: none.

Open questions: none. The PR description explains the one design choice: scan --prune leg (b), where the gem has left the lock, deliberately gets no warning.

Auto-merge is armed, so approving sends this straight to the merge queue.


Generated by Claude Code

Merged via the queue into main with commit dbedeb1 Oct 22, 2026
54 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-gem-vendored-unwind-stale-cache branch October 22, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

3 participants