diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 9b37242d4..a580c6c5e 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -1433,7 +1433,7 @@ command's integration tests check its output. | `maven_trusted_checksums_left` / `nuget_default_config_left` / `upstream_uv_override_removed` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (v5.0): `.mvn` config keeps the trusted-checksums resolver lines because it holds more than hosted mode writes; `nuget.config` now holds only the nuget.org source (delete it if hosted mode created it); a transitive `override-dependencies` entry hosted mode added to `pyproject.toml` (the one under its `# socket-patch hosted` comment) was removed; a user-authored override is never removed and never reported. | | `hosted_resolution_orphaned` | rollback/remove `warnings[]`; list `warnings[]` (human: `Warning: …` on stderr) | yarn berry (#1203): the root `package.json` keeps a Socket-hosted `resolutions` selector that no live `yarn.lock` entry resolves and no lock descriptor matches (left behind by `yarn remove` / `yarn up`). `rollback` / `remove` removed it (the lock is untouched; an emptied `resolutions` object is dropped). uv (#1445): `pyproject.toml` (or a script's PEP 723 block) keeps a hosted `override-dependencies` pin and its `[tool.uv.sources]` URL for a transitive package the lock no longer holds (left behind by `uv remove ` / `uv lock --upgrade-package `). `rollback` / `remove` removed both lines and the lock's `[manifest]` record of them (emptied tables are dropped). `list` only reports either and names the scoped `remove ` that retires it alone. Never flips the exit. | | `upstream_registry_fallback` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore: a yarn berry, pnpm or vlt entry is restored from the version document of the registry the project resolves it against (yarn berry: `npmScopes` / `YARN_NPM_REGISTRY_SERVER` / `npmRegistryServer` from the `.yarnrc.yml` chain, as above, the pnpm lock's sibling `.npmrc` `registry` / `@scope:registry` or pnpm-workspace.yaml `registry` / `registries`, vlt's node registry); that registry could not be read (e.g. it needs credentials) or, for yarn berry, could not be determined (an rc value referencing an environment variable that is set, or an unset one with no default; a flow-style `npmScopes`), so the default registry's document was used and the restored tarball URL may not be the mirror's. | -| `upstream_gem_stale_cache` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (#1260): a restored gem's `-.gem` is still in Bundler's cache dir (`cache_path`, default `vendor/cache`, resolved as for the Gem stale-install guard) and its sha256 is not the upstream one (the restored `CHECKSUMS` entry, else the rubygems.org compact index), or it could not be checked (`--offline`, a registry error). Bundler installs from that dir first, so a `bundle cache` taken while the hosted pin was live makes every later install fail on the upstream checksum (exit 37) or, on bundler < 2.6 frozen installs, keep installing the patched bytes. The detail names the file. Remedy: delete it, then run `bundle cache` to cache the upstream gem in its place (or `bundle install` if the project does not commit its cache; with the cache dir committed, a frozen install reads only the cache). Read-only: the restore never deletes it. Not raised for an archive whose sha256 matches upstream. | +| `upstream_gem_stale_cache` | rollback/remove `warnings[]`; a vendored unwind's rollback `warnings[]`, remove / `vendor --revert` / `vendor` (manifest reconcile) advisory events and `scan --prune` `gc.warnings[]`; vendor advisory event (takeover) | upstream restore (#1260), or a wet vendored gem unwind (`remove`, `rollback`, `vendor --revert`, `scan --prune`, `vendor`'s reconcile) that puts the gem back on its upstream entry (#1594; a gem locked from a remote other than rubygems.org with no `CHECKSUMS` sha is not judged): a restored gem's `-.gem` is still in Bundler's cache dir (`cache_path`, default `vendor/cache`, resolved as for the Gem stale-install guard) and its sha256 is not the upstream one (the restored `CHECKSUMS` entry, else the rubygems.org compact index), or it could not be checked (`--offline`, a registry error). Bundler installs from that dir first, so a `bundle cache` taken while the hosted pin was live makes every later install fail on the upstream checksum (exit 37) or, on bundler < 2.6 frozen installs, keep installing the patched bytes. The detail names the file. Remedy: delete it, then run `bundle cache` to cache the upstream gem in its place (or `bundle install` if the project does not commit its cache; with the cache dir committed, a frozen install reads only the cache). Read-only: the restore never deletes it. Not raised for an archive whose sha256 matches upstream. The takeover's copy adds that the vendored copy is installed instead while the gem stays vendored, and that the vendored unwind names the archive again. | | `upstream_pnpm_tarball_setting_guessed` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (#902): nothing showed which pnpm wrote a hosted `pnpm-lock.yaml` (no unpinned registry entry that shows the setting, no `node_modules/.modules.yaml` install record, no package.json `packageManager` pin; for a Rush lock, no rush.json `pnpmVersion`), so its entries were restored with or without `tarball:` by pnpm 10's reading of `lockfileIncludeTarballUrl` (pnpm-workspace.yaml, else `.npmrc` `lockfile-include-tarball-url`), and pnpm 9 (which reads only `.npmrc`) or pnpm >= 11 (which reads only pnpm-workspace.yaml) would have read it the other way. The detail names the lock, the setting followed, the pnpm that disagrees and the entries. Remedy: pin the pnpm (package.json `packageManager`, or reinstall so the install record names it; rush.json `pnpmVersion` for Rush), or give the two files the same value so every pnpm reads it alike; a rollback or remove can then be redone by restoring the lock from version control and re-running. Not raised when evidence decided, when both files read the same on every pnpm, or when the tarball is one pnpm records regardless. | | `yarn_berry_node_gyp_unresolved` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (#737): yarn's npm resolver gives the restored registry entry an implicit `node-gyp: "npm:latest"` dependency (the version document runs `node-gyp` in a script without declaring it), which the hosted pin dropped because yarn never gives a tarball-locator entry one, and the lock no longer has a `node-gyp@npm:latest` entry (the pin dropped the subtree only node-gyp reached). The entry is restored without it; a plain `yarn install --immutable` accepts that, a hardened or `--refresh-lockfile` install reports the lock modified. Remedy: run `yarn install` once. Not raised while another entry keeps `node-gyp@npm:latest` in the lock: the dependency is then restored too. | | `legacy_redirect_ledger_kept` | rollback `warnings[]` (+ remove stderr) | v5.0: a pre-v5 `.socket/vendor/redirect-state.json` could not be deleted once no hosted pin was left; the file is inert (never read for planning). Never flips the exit. | diff --git a/crates/socket-patch-cli/src/commands/vendor.rs b/crates/socket-patch-cli/src/commands/vendor.rs index 3516e7c49..1da156514 100644 --- a/crates/socket-patch-cli/src/commands/vendor.rs +++ b/crates/socket-patch-cli/src/commands/vendor.rs @@ -640,6 +640,23 @@ pub(crate) fn symlinked_wiring_warnings(cwd: &Path, purl: &str) -> Vec String { + if code == socket_patch_core::patch::redirect::upstream::GEM_STALE_CACHE { + format!( + "{detail} (while the gem stays vendored, bundler installs the vendored copy \ + instead; the command that unvendors it, such as `remove`, `rollback` or \ + `vendor --revert`, names this archive again)" + ) + } else { + detail.to_string() + } +} + pub(crate) fn record_warning( env: &mut Envelope, purl: &str, @@ -3269,7 +3286,7 @@ pub(crate) async fn vendor_records_reusing( let mut advisories: Vec = restore .warnings .iter() - .map(|(code, detail)| VendorWarning::new(code, detail.clone())) + .map(|(code, detail)| VendorWarning::new(code, takeover_detail(code, detail))) .collect(); if common.dry_run { // The refusal the backend would raise over the restored @@ -4507,7 +4524,8 @@ pub(crate) struct VendorGcSummary { } /// The revert warnings `scan --prune` forwards into `gc.warnings[]`: only -/// the "the installed tree still holds the vendored copy" advisories. The +/// the "the installed tree still holds the vendored copy" advisories, and +/// the patched gem archive Bundler's cache would install again (#1594). The /// revert's other warnings are routine for a prune and stay out: /// `vendor_lock_entry_removed` is the normal leg-(b) case (the dependency /// was uninstalled), and a drift keep is already reported through @@ -4516,6 +4534,7 @@ const GC_FORWARDED_ADVISORIES: &[&str] = &[ socket_patch_core::vendor::bun_lock::REINSTALL_REQUIRED, socket_patch_core::vendor::vlt_lock::REINSTALL_REQUIRED, crate::commands::pypi_reinstall::VENDOR_CODE, + socket_patch_core::patch::redirect::upstream::GEM_STALE_CACHE, ]; impl VendorGcSummary { @@ -4627,7 +4646,8 @@ pub(crate) async fn run_vendor_gc( let mut outcome = dispatch_revert_one(&entry, &common.cwd, false).await; // The patch left the manifest while the package stays locked: // the restored lock pins the version the vendored build is - // installed as, which PDM, uv and Pipenv keep (#477). + // installed as, which PDM, uv and Pipenv keep (#477), and a + // patched gem archive in Bundler's cache installs again (#1594). if outcome.success && !outcome.kept_artifact { crate::commands::pypi_reinstall::push_vendor_advisory( &common.cwd, @@ -4636,6 +4656,14 @@ pub(crate) async fn run_vendor_gc( &mut outcome.warnings, ) .await; + crate::commands::vendored_backend::push_gem_stale_cache( + &common.cwd, + &entry.ecosystem, + &entry.base_purl, + common.offline, + &mut outcome.warnings, + ) + .await; } out.take_advisories(&outcome); if !outcome.success { diff --git a/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs b/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs index 0830ec6b1..e8c602557 100644 --- a/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs +++ b/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs @@ -131,6 +131,10 @@ impl<'a> VendoredBackend<'a> { for key in keys { // Captured before a clean revert drops the entry. let flavor = state.entries.get(key).and_then(|e| e.flavor.clone()); + let gem = state + .entries + .get(key) + .map(|e| (e.ecosystem.clone(), e.base_purl.clone())); let mut result = revert_vendor_entry(&self.common.cwd, key, state, opts).await; let hard_failure = matches!( result.step, @@ -154,6 +158,20 @@ impl<'a> VendoredBackend<'a> { &mut result.warnings, ) .await; + // A dry run reads the still-vendored lock, whose bare + // CHECKSUMS entry can't judge the archive: the wet run + // names it. + let dry = matches!(result.step, VendorRevertStep::WouldRevert); + if let Some((ecosystem, purl)) = gem.as_ref().filter(|_| !dry) { + push_gem_stale_cache( + &self.common.cwd, + ecosystem, + purl, + self.common.offline, + &mut result.warnings, + ) + .await; + } } out.push(RevertedEntry { key: key.clone(), @@ -169,6 +187,32 @@ impl<'a> VendoredBackend<'a> { } } +/// Name a patched `-.gem` left in Bundler's cache dir once +/// the vendored ledger entry of `ecosystem` for `base_purl` is unwired +/// (#1594): a `bundle cache` taken while a hosted pin was live keeps it +/// there through the takeover, and bundler installs from it again once the +/// lock is back on upstream. The hosted unwind raises the same warning +/// (#1260). +pub(crate) async fn push_gem_stale_cache( + root: &Path, + ecosystem: &str, + base_purl: &str, + offline: bool, + warnings: &mut Vec, +) { + if socket_patch_core::vendor::jvm::layout::ledger_ecosystem(ecosystem) != "gem" { + return; + } + if let Some((code, detail)) = + socket_patch_core::patch::redirect::upstream::gem_stale_cache_after_unwind( + root, base_purl, offline, + ) + .await + { + warnings.push(VendorWarning::new(code, detail)); + } +} + /// One entry [`VendoredBackend::revert`] handled. pub(crate) struct RevertedEntry { /// The ledger key. diff --git a/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs b/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs index fc0802015..1b8937c21 100644 --- a/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs +++ b/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs @@ -3689,3 +3689,223 @@ async fn gem_hosted_unwind_names_a_patched_archive_in_vendor_cache() { unwind_names_the_patched_cached_archive(&fx, "rollback"); unwind_names_the_patched_cached_archive(&fx, "remove"); } + +/// The vendored half of the patch API for the fixture's patch: the +/// patched `.gem` (the last `build_gem` of `DEP` left it in the stage dir) +/// and the stub gemspec `gem spec --ruby` derives from it, beside the +/// hosted registry override. Priority 1, so it answers every later +/// `patches/package` call in place of the hosted-only grant. +async fn mount_vendored_grant(fx: &RedirectFixture) { + use base64::Engine as _; + use sha2::Sha512; + let server = &fx._server; + let gem_path = fx + .tmp + .path() + .join("gem-stage") + .join(format!("{DEP}-src")) + .join(format!("{DEP}-{DEP_VERSION}.gem")); + let gem = std::fs::read(&gem_path).expect("the patched .gem from build_gem"); + let mut spec = Command::new("gem"); + spec.args(["spec", gem_path.to_str().unwrap(), "--ruby"]); + cache_env::isolate(&mut spec); + let stub = spec.output().expect("failed to run gem spec"); + assert!( + stub.status.success(), + "gem spec --ruby:\n{}", + String::from_utf8_lossy(&stub.stderr) + ); + let stub = stub.stdout; + let sri = |bytes: &[u8]| { + format!( + "sha512-{}", + base64::engine::general_purpose::STANDARD.encode(Sha512::digest(bytes)) + ) + }; + let base = format!("/artifacts/{UUID}"); + let gem_url = format!("{}{base}/{DEP}-{DEP_VERSION}.gem", server.uri()); + let stub_url = format!("{}{base}/{DEP}-{DEP_VERSION}.gemspec", server.uri()); + let patched_sha = sha256_hex(&gem); + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG}/patches/package"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "results": { + UUID: { + "status": "granted", + "url": gem_url, + "purl": PURL, + "artifacts": [ + { "kind": "tarball", "url": gem_url, + "integrity": { "sha512": sri(&gem), "sha256": patched_sha } }, + { "kind": "gem-stub-gemspec", "url": stub_url, + "integrity": { "sha512": sri(&stub) } } + ], + "registryOverride": { + "kind": "rubygems-compact-index", + "indexUrl": fx.index_url, + "identifiers": { + "name": DEP, + "version": DEP_VERSION, + "gemChecksumSha256": patched_sha, + } + } + } + } + }))) + .with_priority(1) + .mount(server) + .await; + for (leaf, bytes) in [ + (format!("{DEP}-{DEP_VERSION}.gem"), gem), + (format!("{DEP}-{DEP_VERSION}.gemspec"), stub), + ] { + Mock::given(method("GET")) + .and(path(format!("{base}/{leaf}"))) + .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes)) + .mount(server) + .await; + } +} + +/// #1594 through one vendored unwind `command`: the hosted pin was live +/// when the project ran `bundle cache`, so `vendor/cache` holds the PATCHED +/// archive. A hosted → vendored takeover wires the gem through its path +/// source (the archive is unused while vendored), and the later unwind puts +/// the pair back on the upstream `GEM` entry, where bundler installs from +/// that archive first. The unwind must name the file, exactly as the hosted +/// unwind does (#1260). +fn vendored_unwind_names_the_patched_cached_archive(fx: &RedirectFixture, command: &str) { + let dir = stage_fresh_checkout(fx, &format!("vendored-cache-{command}")); + let install = bundle(&dir, &["install"]); + assert!( + install.status.success(), + "{command}: hosted install:\n{}", + String::from_utf8_lossy(&install.stderr) + ); + let cache_cmd = if fx.bundler.at_least(2, 0) { + "cache" + } else { + "package" + }; + let cache = bundle(&dir, &[cache_cmd]); + assert!( + cache.status.success(), + "{command}: bundle {cache_cmd}:\n{}", + String::from_utf8_lossy(&cache.stderr) + ); + let archive = dir + .join("vendor") + .join("cache") + .join(format!("{DEP}-{DEP_VERSION}.gem")); + assert!( + archive.is_file(), + "{command}: bundle {cache_cmd} wrote no archive" + ); + + let api = fx._server.uri(); + let upstream = format!("{api}/upstream"); + let cwd = dir.to_str().expect("utf8 tmp path"); + let common = [ + "--json", + "--cwd", + cwd, + "--api-url", + &api, + "--org", + ORG, + "--api-token", + "fake", + "--patch-server-url", + &api, + ]; + let envs = [("SOCKET_RUBYGEMS_URL", upstream.as_str())]; + + let mut takeover = vec!["get", UUID, "--mode", "vendored", "--yes"]; + takeover.extend(common); + let (code, stdout, stderr) = run_socket_env(&dir, &takeover, &envs); + assert_eq!( + code, 0, + "{command}: takeover:\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let gemfile = std::fs::read_to_string(dir.join(fx.gemfile_name)).unwrap(); + assert!( + gemfile.contains(".socket/vendor/gem/"), + "{command}: the takeover vendored the gem:\n{gemfile}" + ); + // The takeover's own notice says the vendored copy shadows the archive + // for now, not that installs already fail. + let env: serde_json::Value = serde_json::from_str(&stdout).unwrap(); + let notice = env["events"] + .as_array() + .into_iter() + .flatten() + .find(|e| e["errorCode"] == "upstream_gem_stale_cache") + .unwrap_or_else(|| panic!("{command}: the takeover names the archive: {env}")); + assert!( + notice["reason"] + .as_str() + .is_some_and(|r| r.contains("while the gem stays vendored")), + "{command}: {notice}" + ); + + let mut argv: Vec<&str> = match command { + "vendor --revert" => vec!["vendor", "--revert", "--yes"], + "remove" => vec!["remove", PURL, "--yes"], + _ => vec![command, PURL], + }; + argv.extend(common); + let (code, stdout, stderr) = run_socket_env(&dir, &argv, &envs); + let env: serde_json::Value = serde_json::from_str(&stdout).unwrap_or_else(|e| { + panic!("{command}: not JSON: {e}\nstdout:\n{stdout}\nstderr:\n{stderr}") + }); + assert_eq!(code, 0, "{command}: {env}\nstderr:\n{stderr}"); + let gemfile = std::fs::read_to_string(dir.join(fx.gemfile_name)).unwrap(); + assert!( + !gemfile.contains(".socket/vendor/gem/"), + "{command}: the gem is no longer vendored:\n{gemfile}" + ); + let hits: Vec = env["warnings"] + .as_array() + .into_iter() + .flatten() + .filter(|w| w["code"] == "upstream_gem_stale_cache") + // `remove` and `vendor --revert` report a vendored unwind's + // advisories as uncounted events, `rollback` in `warnings[]`. + .chain( + env["events"] + .as_array() + .into_iter() + .flatten() + .filter(|e| e["errorCode"] == "upstream_gem_stale_cache"), + ) + .map(|w| w.to_string()) + .collect(); + assert_eq!(hits.len(), 1, "{command}: one stale-cache warning: {env}"); + assert!( + hits[0].contains(&archive.display().to_string()), + "{command}: the warning names the cached archive: {}", + hits[0] + ); +} + +#[tokio::test(flavor = "multi_thread")] +#[ignore = "host capstone: shells out to a real ruby/gem/bundler (>= 1.17); the unpinned `test` \ + job skips it, an e2e job with a pinned toolchain runs it via --ignored"] +async fn gem_vendored_unwind_names_a_patched_archive_in_vendor_cache() { + let Some(fx) = redirect_scanned_project( + "vendored-cache-unwind", + Spelling::Gemfile, + false, + true, + None, + Driver::ScanVex, + ) + .await + else { + return; + }; + mount_vendored_grant(&fx).await; + for command in ["remove", "rollback", "vendor --revert"] { + vendored_unwind_names_the_patched_cached_archive(&fx, command); + } +} diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/client.rs b/crates/socket-patch-core/src/patch/redirect/upstream/client.rs index 9313d6f51..9af627e3e 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/client.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/client.rs @@ -83,7 +83,7 @@ fn crates_index_path(name: &str) -> String { /// The RubyGems compact index root; override with `SOCKET_RUBYGEMS_URL`. pub(crate) const DEFAULT_RUBYGEMS: &str = "https://rubygems.org"; -fn rubygems_base() -> String { +pub(super) fn rubygems_base() -> String { std::env::var("SOCKET_RUBYGEMS_URL") .ok() .map(|v| v.trim().trim_end_matches('/').to_string()) diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/gem.rs b/crates/socket-patch-core/src/patch/redirect/upstream/gem.rs index 67d078cf2..9adc940fd 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/gem.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/gem.rs @@ -58,7 +58,7 @@ use std::collections::{BTreeMap, BTreeSet}; use regex::Regex; -use super::{Ctx, FormatResult, HostedPin, View}; +use super::{Ctx, FormatResult, HostedPin, UpstreamClient, View}; use crate::formats::gem::{ bundler_manifest_for, parse_spec, same_remote, split_checksum_entry, BUNDLER_LOCKS, }; @@ -1276,7 +1276,18 @@ pub(crate) async fn restore( { continue; } - if let Some(warning) = stale_cache_warning(view.root(), pin, lock.as_deref(), ctx).await + let Some((name, version)) = pin.name_version() else { + continue; + }; + if let Some(warning) = stale_cache_warning( + view.root(), + &pin.purl, + &name, + &version, + lock.as_deref(), + ctx.client, + ) + .await { result.warnings.push(warning); } @@ -1329,20 +1340,21 @@ fn lock_checksum(lock: &str, name: &str, version: &str) -> Option { /// deleted. async fn stale_cache_warning( root: &std::path::Path, - pin: &HostedPin, + purl: &str, + name: &str, + version: &str, lock: Option<&str>, - ctx: &Ctx<'_>, + client: &UpstreamClient, ) -> Option<(&'static str, String)> { - let (name, version) = pin.name_version()?; let cache_dir = crate::crawlers::ruby_crawler::bundler_app_cache_dir(root).await; let archive = cache_dir.join(format!("{name}-{version}.gem")); if !archive.is_file() { return None; } let got = crate::vendor::file_sha256_hex(&archive).await; - let upstream = match lock.and_then(|l| lock_checksum(l, &name, &version)) { + let upstream = match lock.and_then(|l| lock_checksum(l, name, version)) { Some(sha) => Ok(sha), - None => ctx.client.rubygems_sha256(&name, &version).await, + None => client.rubygems_sha256(name, version).await, }; let fix = "delete it, then run `bundle cache` to cache the upstream gem in its place \ (or `bundle install` if the project does not commit its cache)"; @@ -1353,7 +1365,6 @@ async fn stale_cache_warning( holds a non-upstream (patched) archive at {path}; bundler installs from that \ dir first, so installs fail on the upstream checksum or keep the patched \ bytes: {fix}", - purl = pin.purl, path = archive.display(), ), (_, why) => format!( @@ -1361,7 +1372,6 @@ async fn stale_cache_warning( {path}, which could not be checked against the upstream sha256{why}; if it \ was cached while the hosted patch was live, bundler keeps installing it: \ {fix}", - purl = pin.purl, path = archive.display(), why = match why { Err(e) => format!(" ({e})"), @@ -1372,6 +1382,56 @@ async fn stale_cache_warning( Some((STALE_CACHE, detail)) } +/// [`STALE_CACHE`] for a gem a vendored unwind (`remove`, `rollback`, +/// `vendor --revert`, `scan --prune`, `vendor`'s reconcile) put back on its +/// upstream `GEM` entry at `root` (#1594). The archive a `bundle cache` +/// took while a hosted pin was live survives a hosted → vendored takeover +/// (the path source shadows it), so once the vendored copy is unwired +/// bundler installs from it again. Judged like the hosted restore's check, +/// against the lock's `CHECKSUMS` entry, else the rubygems.org compact +/// index (unless `offline`). A gem locked from another remote with no +/// `CHECKSUMS` sha can't be judged, and is most likely a private gem cached +/// on purpose: no warning. +pub(super) async fn unwound_stale_cache( + root: &std::path::Path, + purl: &str, + offline: bool, +) -> Option<(&'static str, String)> { + let (_, name, version) = crate::utils::purl::purl_parts(purl)?; + let mut lock = None; + for file in BUNDLER_LOCKS { + // Guarded: a FIFO or device squatting the lock path must not block. + if let Ok(text) = crate::utils::fs::read_regular_to_string(&root.join(file)).await { + lock = Some(text); + break; + } + } + if let Some(text) = &lock { + if lock_checksum(text, &name, &version).is_none() && !rubygems_locked(text, &name, &version) + { + return None; + } + } + let client = UpstreamClient::new(offline); + stale_cache_warning(root, purl, &name, &version, lock.as_deref(), &client).await +} + +/// Whether `lock` resolves `name (version)` from a `GEM` section whose +/// remote is rubygems.org (or the `SOCKET_RUBYGEMS_URL` stand-in). +fn rubygems_locked(lock: &str, name: &str, version: &str) -> bool { + let lines: Vec<&str> = lock.split('\n').collect(); + let base = super::rubygems_base(); + gem_sections(&lines).iter().any(|sec| { + sec.entries + .iter() + .any(|e| e.name == name && e.version == version && !e.platform) + && sec + .remotes + .iter() + .any(|(_, r)| same_remote(r, RUBYGEMS_REMOTE) || same_remote(r, &base)) + }) +} + /// Restore one gem in a lock + manifest pair: `Ok(None)` when neither wires /// it, else the next `(lock, manifest)` texts. async fn restore_one( @@ -2523,6 +2583,129 @@ mod tests { assert!(hits[0].contains("could not be checked"), "{}", hits[0]); } + /// The project a vendored unwind leaves behind (#1594): the lock back on + /// the upstream `GEM` entry, `checksum` its `CHECKSUMS` sha (none for a + /// pre-2.6 lock), and `archive` in `vendor/cache`. + fn unwound_project( + remote: &str, + checksum: Option<&str>, + archive: Option<&[u8]>, + ) -> tempfile::TempDir { + let tmp = tempfile::tempdir().unwrap(); + let checksums = checksum + .map(|sha| format!("CHECKSUMS\n rails (7.0.0) sha256={sha}\n\n")) + .unwrap_or_default(); + std::fs::write( + tmp.path().join("Gemfile.lock"), + format!( + "GEM\n remote: {remote}\n specs:\n rails (7.0.0)\n\n\ + PLATFORMS\n ruby\n\nDEPENDENCIES\n rails (= 7.0.0)\n\n{checksums}\ + BUNDLED WITH\n 2.6.9\n" + ), + ) + .unwrap(); + if let Some(bytes) = archive { + std::fs::create_dir_all(tmp.path().join("vendor/cache")).unwrap(); + std::fs::write(tmp.path().join("vendor/cache/rails-7.0.0.gem"), bytes).unwrap(); + } + tmp + } + + /// #1594: a hosted → vendored takeover keeps the patched archive a + /// `bundle cache` took while the hosted pin was live, and the vendored + /// unwind puts the lock back on upstream: name the archive, exactly as + /// the hosted unwind does. + #[tokio::test] + async fn vendored_unwind_names_a_patched_archive_in_vendor_cache() { + let upstream_sha = sha_hex(b"upstream gem"); + let tmp = unwound_project( + "https://rubygems.org/", + Some(&upstream_sha), + Some(b"patched gem"), + ); + let (code, detail) = unwound_stale_cache(tmp.path(), "pkg:gem/rails@7.0.0", true) + .await + .expect("the patched archive is named"); + assert_eq!(code, STALE_CACHE); + let path = tmp + .path() + .join("vendor") + .join("cache") + .join("rails-7.0.0.gem"); + assert!(detail.contains(&path.display().to_string()), "{detail}"); + assert!(detail.contains("pkg:gem/rails@7.0.0"), "{detail}"); + assert!(detail.contains("non-upstream"), "{detail}"); + } + + /// #1594: an upstream archive, or none at all, is healthy. + #[tokio::test] + async fn vendored_unwind_keeps_quiet_about_an_upstream_or_missing_archive() { + let upstream_sha = sha_hex(b"upstream gem"); + let tmp = unwound_project( + "https://rubygems.org/", + Some(&upstream_sha), + Some(b"upstream gem"), + ); + let hit = unwound_stale_cache(tmp.path(), "pkg:gem/rails@7.0.0", true).await; + assert_eq!(hit, None); + let tmp = unwound_project("https://rubygems.org/", Some(&upstream_sha), None); + let hit = unwound_stale_cache(tmp.path(), "pkg:gem/rails@7.0.0", true).await; + assert_eq!(hit, None); + } + + /// #1594 on a lock without `CHECKSUMS`: offline, the upstream sha can't + /// be fetched, so the archive is named as unchecked rather than passed. + #[tokio::test] + async fn vendored_unwind_names_an_unchecked_archive_offline() { + let tmp = unwound_project("https://rubygems.org/", None, Some(b"patched gem")); + let (code, detail) = unwound_stale_cache(tmp.path(), "pkg:gem/rails@7.0.0", true) + .await + .expect("the unchecked archive is named"); + assert_eq!(code, STALE_CACHE); + assert!(detail.contains("could not be checked"), "{detail}"); + } + + /// #1594: a FIFO at the lock path is skipped, never opened for reading + /// (which would block): the check goes on without a lock. + #[cfg(unix)] + #[tokio::test] + async fn vendored_unwind_skips_a_fifo_lock() { + let tmp = unwound_project("https://rubygems.org/", None, Some(b"patched gem")); + let lock = tmp.path().join("Gemfile.lock"); + std::fs::remove_file(&lock).unwrap(); + let status = std::process::Command::new("mkfifo") + .arg(&lock) + .status() + .unwrap(); + assert!(status.success()); + let hit = tokio::time::timeout( + std::time::Duration::from_secs(10), + unwound_stale_cache(tmp.path(), "pkg:gem/rails@7.0.0", true), + ) + .await + .expect("a FIFO lock must not block the check"); + assert!(hit.is_some_and(|(_, d)| d.contains("could not be checked"))); + } + + /// #1594: a gem locked from a private remote with no `CHECKSUMS` sha + /// can't be checked against rubygems.org (it isn't from there), and a + /// committed cache of private gems is normal: no warning. + #[tokio::test] + async fn vendored_unwind_skips_an_unchecked_private_gem() { + let tmp = unwound_project("https://gems.example.com/", None, Some(b"private gem")); + let hit = unwound_stale_cache(tmp.path(), "pkg:gem/rails@7.0.0", true).await; + assert_eq!(hit, None); + // A CHECKSUMS sha still judges it, whatever the remote. + let upstream_sha = sha_hex(b"upstream gem"); + let tmp = unwound_project( + "https://gems.example.com/", + Some(&upstream_sha), + Some(b"patched gem"), + ); + let hit = unwound_stale_cache(tmp.path(), "pkg:gem/rails@7.0.0", true).await; + assert!(hit.is_some_and(|(_, d)| d.contains("non-upstream"))); + } + /// A gem restored in both a `Gemfile` and a `gems.rb` pair shares one /// cache dir: one warning, not one per pair. #[tokio::test] diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs b/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs index 7d09cc4e3..a7f1b615d 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs @@ -42,6 +42,7 @@ mod sbt; mod uv; mod vlt; +use client::rubygems_base; pub(crate) use client::UpstreamClient; pub(crate) use uv::{declared_lock_specifier, respell_lock_specifier, LockRequirementArray}; @@ -835,6 +836,24 @@ fn format_of(rel: &str) -> Format { } } +/// The warning code for a patched gem archive left in Bundler's cache dir +/// once its gem is back on the upstream registry entry (#1260, #1594). +pub const GEM_STALE_CACHE: &str = gem::STALE_CACHE; + +/// The [`GEM_STALE_CACHE`] advisory, as `(code, detail)`, for the gem +/// `purl` a vendored unwind just put back on its upstream entry in the +/// project at `root` (#1594). `None` when Bundler's cache dir holds no +/// archive for it, or holds the upstream one. `offline` skips the +/// rubygems.org lookup a lock without `CHECKSUMS` needs, so the archive is +/// then named as unchecked. +pub async fn gem_stale_cache_after_unwind( + root: &Path, + purl: &str, + offline: bool, +) -> Option<(&'static str, String)> { + gem::unwound_stale_cache(root, purl, offline).await +} + /// Restore every pin in `pins` to its default upstream registry entry. /// See the module docs for the contract. pub async fn restore_upstream(