diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 5f9403939..99aad9fd0 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -579,8 +579,8 @@ A reverted lockfile plus a leftover ledger or artifact therefore stops attesting | `patched_ref_invalid` | A Socket-shaped reference failed validation or is not live wiring: unsafe coordinates, a non-canonical uuid, a leaf naming another package, a path escaping the root, an inert or orphaned entry. | | `patched_ref_unattributable` | A Socket patch uuid cannot be tied to exactly one artifact / version. Examples: a maven repository no pin names, a nuget source without a mapping, a lock contested by another lock. | | `sbt_owned_file_modified` | `socket-patch.sbt` / `socket-patch-vendor.sbt` is not byte-for-byte what socket-patch generates; none of its pins is discovered. | -| `sbt_resolution_unverified` | An sbt pin the build's local resolution evidence (`target/`, after `sbt update`) does not verify: the reference stays, without the lockfile basis (a hosted pin the build provably bypasses is also omitted, `vex_sbt_pin_bypassed`). | -| `vendored_tree_missing` | A `socket-patch-vendor.sbt` pin whose `.socket/vendor/maven2/` file is missing or modified. | +| `sbt_resolution_unverified` | An sbt pin the build's local resolution evidence (`target/`, after `sbt update`) does not verify, or a build-wide blocker applies (stale or partial evidence, project definitions that cannot be read statically, a `dependencyOverrides` / `onLoad` / `resolvers` reassignment, a `build.sbt.lock`); the detail names the cause and its remedy, or that it has none. A hosted reference stays, without the lockfile basis (a hosted pin the build provably bypasses is also omitted, `vex_sbt_pin_bypassed`); a vendored pin is not attested (`vex_sbt_resolution_unverified`). | +| `vendored_tree_missing` | A `socket-patch-vendor.sbt` pin whose `.socket/vendor/maven2/` file is missing or modified; it is not attested (`vex_sbt_vendored_tree_missing`). | Human mode also prints `Note:` lines: superseded records, fetch failures, `--offline` withholding fetches, conflict details, and why a claim is dead. While patch records are fetched, a terminal shows a transient `Fetching patch records... (n/N)` status line on stderr (never under `--json` / `--silent`). @@ -2689,6 +2689,8 @@ match it, withholds the statement: | `vex_maven_reactor_root` | run warning and `failed[].reason` | A hosted Maven pin sits in a root `pom.xml` that declares `` / `` (a profile's included), where a module's own literal `` overrides it; only the root pom is read, so no statement (#261). `list`, `rollback` and `remove` still find the pin. | | `go_workspace_override` | run warning and `failed[].reason` | A hosted or vendored Go `replace` is wired, but the workspace go resolves (an ancestor's `go.work`, one `GOWORK` names, members outside the project included) overrides, ignores or rejects it, or go builds another `go.work` than the root's that holds it; no statement while it does. `list`, `rollback` and `remove` still find the replace. | | `vex_maven_classifier_unpatched` | run warning and `failed[].reason` | A hosted Maven pin's GA also has a live executable `` copy (not `sources` / `javadoc`) at a version other than the pin, which the classifier-less pin does not retarget and the grant does not serve; no statement while it does. `list`, `rollback` and `remove` still find the pin. | +| `vex_sbt_resolution_unverified` | run warning and `failed[].reason` | A vendored sbt pin (`socket-patch-vendor.sbt`) whose resolution the build's local evidence does not verify, by the same check and build-wide blockers as a hosted sbt pin (no evidence or evidence older than a build source, a declared project with none, a `dependencyOverrides` / `resolvers` reassignment, a `build.sbt.lock`, project definitions that cannot be read statically, another version or other bytes resolved); no statement while it holds. The note names the cause and its remedy: `sbt update` clears stale or missing evidence, keeping the setting's existing entries (appending with `+=` / `++=` in place of a `:=`, `~=` or `--=`) clears a reassignment, removing what forces another version clears a version conflict, and a build whose projects cannot be read statically (a computed `Project(...)`, `projectMatrix`, `crossProject`) is never attested. `vendor --check` and `scan` still treat the wiring as live. | +| `vex_sbt_vendored_tree_missing` | run warning and `failed[].reason` | A vendored sbt pin whose committed `.socket/vendor/maven2/` file is missing or does not hash to the pinned sha256 (sbt's load-time check fails the build); no statement until the tree is restored. | | `vex_nuget_lock_other_version` | run warning and `failed[].reason` | A NuGet Socket source maps an id that a governed `packages.lock.json` also locks at a version other than the patched one; the mapping routes every version to a source that serves only the patched one, so that target framework cannot restore, and both writers refuse such a lock. No statement until every target framework locks only the patched version. `list`, `rollback` and `remove` still find the wiring. | | `vex_sbt_pin_bypassed` | run warning and `failed[].reason` | A hosted sbt pin (`socket-patch.sbt`) whose bytes the build provably does not use, and that no current evidence verifies: a build source reassigns `dependencyOverrides` (replacing the generated override) or reassigns `onLoad` without keeping its previous value (dropping the load-time verifier and resolver order), or a resolution record newer than the generated files and every build source shows the GA's base version, or other bytes at the pinned version from outside the pin repository. The copy downloaded into `.socket/sbt-hosted` then proves nothing, so no statement. `list`, `rollback` and `remove` still find the pin. | | `vex_gradle_derived_cache_unchecked` | run warning | The derived-cache walk was cut short (a very large transforms cache); the statement is still emitted, the copies the walk did reach were checked. | diff --git a/crates/socket-patch-cli/src/commands/vex_sources.rs b/crates/socket-patch-cli/src/commands/vex_sources.rs index 65948a457..0e24fe6f9 100644 --- a/crates/socket-patch-cli/src/commands/vex_sources.rs +++ b/crates/socket-patch-cli/src/commands/vex_sources.rs @@ -232,6 +232,14 @@ const NOTE_MAVEN_REACTOR_ROOT: &str = "vex_maven_reactor_root"; /// classifier copy at another version, which the pin does not reach /// (`vex::Unattested`). const NOTE_MAVEN_CLASSIFIER_UNPATCHED: &str = "vex_maven_classifier_unpatched"; +/// Omission tag and note: a vendored sbt pin the build's local resolution +/// evidence does not verify, or under a build-wide blocker +/// (`vex::Unattested`, `UnattestedKind::SbtResolutionUnverified`). +const NOTE_SBT_RESOLUTION_UNVERIFIED: &str = "vex_sbt_resolution_unverified"; +/// Omission tag and note: a vendored sbt pin's committed tree file is +/// missing or modified (`vex::Unattested`, +/// `UnattestedKind::SbtVendoredTreeMissing`). +const NOTE_SBT_VENDORED_TREE_MISSING: &str = "vex_sbt_vendored_tree_missing"; /// Omission tag and note: a NuGet Socket source's id is also locked at a /// version other than the patched one, which that source cannot serve /// (`vex::Unattested`). @@ -284,6 +292,17 @@ fn unattested_note(kind: UnattestedKind) -> (&'static str, &'static str) { "not attested while that classifier copy resolves an unpatched version; the \ wiring itself is intact, so re-running `scan` does not change this", ), + UnattestedKind::SbtResolutionUnverified => ( + NOTE_SBT_RESOLUTION_UNVERIFIED, + "not attested while that holds (the detail names the remedy, or that there is \ + none); the wiring itself is intact, so `vendor --check` and `scan` still treat it \ + as live", + ), + UnattestedKind::SbtVendoredTreeMissing => ( + NOTE_SBT_VENDORED_TREE_MISSING, + "not attested while the committed tree differs from the pin (restore \ + `.socket/vendor/maven2` from version control, or re-run `socket-patch vendor`)", + ), UnattestedKind::NugetLockOtherVersion => ( NOTE_NUGET_LOCK_OTHER_VERSION, "not attested until every target framework locks only the patched version (align \ @@ -303,6 +322,39 @@ fn unattested_note(kind: UnattestedKind) -> (&'static str, &'static str) { } } +/// The [`Unattested`](socket_patch_core::vex::Unattested) mark discovery +/// put on patch `uuid` for `purl`'s package, if any. +fn unattested_mark<'d>( + discovery: &'d Discovery, + purl: &str, + uuid: &str, +) -> Option<&'d socket_patch_core::vex::Unattested> { + let pkg = canonical_base_purl(purl); + discovery + .unattested + .iter() + .find(|u| u.uuid == uuid && PurlKey::same(&u.purl, &pkg)) +} + +/// Omit `key` for the mark `u`: its `failed[]` entry and the note naming +/// why and the remedy. +fn gate_unattested( + key: &str, + u: &socket_patch_core::vex::Unattested, + gated: &mut Vec, + notes: &mut Vec, +) { + let (code, remedy) = unattested_note(u.kind); + gated.push(failed(key, code)); + notes.push(PlanNote { + code, + detail: format!( + "{key}: patch {} is wired, but {}; {remedy}", + u.uuid, u.detail + ), + }); +} + /// One hosted-basis view purl's wiring (see [`Plan::hosted`]). #[derive(Debug, Clone, PartialEq, Eq)] pub(crate) struct HostedWiring { @@ -496,15 +548,7 @@ pub(crate) async fn plan( first } }; - let (code, remedy) = unattested_note(u.kind); - gated.push(failed(&c.key, code)); - notes.push(PlanNote { - code, - detail: format!( - "{}: patch {} is wired, but {}; {remedy}", - c.key, c.uuid, u.detail - ), - }); + gate_unattested(&c.key, u, &mut gated, &mut notes); false }); for (key, old_uuid, wired) in &superseded { @@ -537,6 +581,7 @@ pub(crate) async fn plan( continue; } } + let mut took_over = false; let basis = if let Some(vref) = cand .discovered .iter() @@ -573,6 +618,7 @@ pub(crate) async fn plan( cand.uuid = hosted.uuid.clone(); cand.record = Some(hosted); cand.vendor_entry = None; + took_over = true; Basis::Hosted { lockfile_basis: false, } @@ -608,6 +654,21 @@ pub(crate) async fn plan( } else { Basis::Installed }; + // The vendored basis attests the ENTRY's patch, which need not be + // the candidate's uuid (a manifest record under another uuid keeps + // the key; a vendored sbt pin has no ref to swap it in): its mark + // is checked against what is actually attested. + // Likewise the hosted-over-vendored takeover attests the redirect + // ledger's uuid, swapped in after the early gate ran. + let attested = match &basis { + Basis::Vendored(entry) => Some((entry.base_purl.as_str(), entry.uuid.as_str())), + Basis::Hosted { .. } if took_over => Some((cand.key.as_str(), cand.uuid.as_str())), + _ => None, + }; + if let Some(u) = attested.and_then(|(purl, uuid)| unattested_mark(&discovery, purl, uuid)) { + gate_unattested(&cand.key, u, &mut gated, &mut notes); + continue; + } based.push((cand, basis)); } @@ -1627,6 +1688,29 @@ mod tests { let dead = plan(&common(tmp.path()), mk(), &[], &RunApiClient::new()).await; assert!(dead.view.patches.is_empty()); assert_eq!(dead.gated, vec![failed(key, VENDOR_UNWIRED)]); + + // The takeover attests the redirect ledger's uuid, swapped in after + // the early mark gate ran: a mark on it still gates. + let mut marked = mk(); + marked + .discovery + .unattested + .push(socket_patch_core::vex::Unattested { + purl: key.into(), + uuid: U2.into(), + file: "package-lock.json".into(), + detail: "package-lock.json bypasses it".into(), + kind: UnattestedKind::LockAboveBase, + }); + let gated = plan( + &common(tmp.path()), + marked, + &[key.to_string()], + &RunApiClient::new(), + ) + .await; + assert!(gated.view.patches.is_empty(), "{:?}", gated.view.patches); + assert_eq!(gated.gated, vec![failed(key, NOTE_LOCK_ABOVE_BASE)]); } /// Core discover rule 11 through the plan: a vendor ledger entry and a diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/sbt_vendored.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/sbt_vendored.rs index 535f44529..5e79b5e8c 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/sbt_vendored.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/sbt_vendored.rs @@ -5,8 +5,9 @@ //! //! The suffixed tree path carries no full uuid, so the generated file is no //! `.socket/vendor///` reference: a vendored sbt pin is attested -//! through the vendor ledger `vendor` writes, and the extractor diagnoses -//! what would make that attestation a lie. Each cell vendors for real +//! through the vendor ledger `vendor` writes, and only when the extractor +//! finds nothing that would make that attestation a lie (the hosted sbt +//! evidence gate). Each cell vendors for real //! (`vendor` over the sbt 1.13.0 probe build, the installed jar in an //! isolated Coursier cache), drops `.socket/manifest.json`, and runs //! `vex --offline` (the ledger embeds the record): @@ -14,12 +15,14 @@ //! | cell | state | expected | //! |---|---|---| //! | a | after `sbt update` (the evidence resolves the suffixed version from the tree) | attested, no sbt diagnostic | -//! | b | before `sbt update` (the evidence still resolves the base version) | `sbt_resolution_unverified` | +//! | b | before `sbt update` (the evidence still resolves the base version) | `sbt_resolution_unverified`, `vex_sbt_resolution_unverified`, not attested | //! | c | a tree jar tampered | `vendored_tree_missing`, not attested | //! | d | the generated file edited | `sbt_owned_file_modified` | //! | e | no ledger | nothing attested; the tree check still runs | +//! | f | after `sbt update`, then a `dependencyOverrides :=` | `vex_sbt_resolution_unverified` naming it, not attested | +//! | g | before `sbt update`, the manifest back (same or another uuid) | `vex_sbt_resolution_unverified`, not attested | -use crate::common::envelope::all_codes; +use crate::common::envelope::{all_codes, warning_codes}; use crate::common::git_sha256; use std::path::{Path, PathBuf}; @@ -42,6 +45,8 @@ const CENTRAL: &str = "repo1.maven.org/maven2"; const MEMBER: &str = "META-INF/NOTICE.txt"; const VULN: &str = "GHSA-sbtv-test-xxxx"; const VENDORED_FILE: &str = "socket-patch-vendor.sbt"; +/// A manifest uuid other than the vendored one. +const OTHER_UUID: &str = "9e8d7c6b-5a49-4382-9f0e-2a3b4c5d6e7f"; fn tree_jar(proj: &Path) -> PathBuf { proj.join(".socket/vendor/maven2/com/google/code/gson/gson") @@ -69,6 +74,8 @@ struct Fx { _tmp: tempfile::TempDir, proj: PathBuf, home: SbtHome, + /// The manifest `vendor` ran with. + manifest: Value, } impl Fx { @@ -111,6 +118,7 @@ impl Fx { _tmp: tmp, proj, home, + manifest, }; let (code, env) = fx.run(&["vendor"]); assert_eq!(code, Some(0), "vendor: {env}"); @@ -120,6 +128,17 @@ impl Fx { fx } + /// Put the manifest back, its gson record filed under `uuid`. + fn restore_manifest(&self, uuid: &str) { + let mut manifest = self.manifest.clone(); + manifest["patches"][GSON.purl()]["uuid"] = uuid.into(); + std::fs::write( + self.proj.join(".socket/manifest.json"), + serde_json::to_vec_pretty(&manifest).unwrap(), + ) + .unwrap(); + } + /// What `sbt update` records after vendoring. fn sbt_update(&self) { let changed = record_pinned_resolution( @@ -197,6 +216,17 @@ fn attested(doc: Option<&Value>) -> bool { }) } +/// The details of the top-level warnings coded `code`. +fn warning_details(env: &Value, code: &str) -> Vec { + env["warnings"] + .as_array() + .into_iter() + .flatten() + .filter(|w| w["code"] == code) + .filter_map(|w| w["detail"].as_str().map(str::to_string)) + .collect() +} + const SBT_DIAGS: &[&str] = &[ "sbt_resolution_unverified", "vendored_tree_missing", @@ -220,11 +250,66 @@ fn sbt_vendored_vex_attests_after_sbt_update() { #[test] fn sbt_vendored_vex_before_sbt_update_is_unverified() { let fx = Fx::vendored(); - let (_, env, _) = fx.vex(); + let (_, env, doc) = fx.vex(); + let got = all_codes(&env); + assert!( + got.contains(&"sbt_resolution_unverified".to_string()), + "{env}" + ); + assert!( + got.contains(&"vex_sbt_resolution_unverified".to_string()), + "{env}" + ); + assert!( + !attested(doc.as_ref()), + "an unverified resolution is never attested: {doc:?}" + ); +} + +#[test] +fn sbt_vendored_vex_withholds_under_an_overrides_reassignment() { + let fx = Fx::vendored(); + fx.sbt_update(); + // Dated back to its old time, the edit stays older than the evidence + // however slow the run (staleness would withhold the pin too). + let build = fx.proj.join("build.sbt"); + let dated = std::fs::metadata(&build).unwrap().modified().unwrap(); + let mut text = std::fs::read_to_string(&build).unwrap(); + text.push_str("ThisBuild / dependencyOverrides := Seq()\n"); + std::fs::write(&build, text).unwrap(); + std::fs::File::options() + .write(true) + .open(&build) + .unwrap() + .set_modified(dated) + .unwrap(); + let (_, env, doc) = fx.vex(); + let details = warning_details(&env, "vex_sbt_resolution_unverified"); assert!( - all_codes(&env).contains(&"sbt_resolution_unverified".to_string()), + details.iter().any(|d| d.contains("`dependencyOverrides`")), "{env}" ); + assert!( + !attested(doc.as_ref()), + "an overrides reassignment replaces the pin: {doc:?}" + ); +} + +/// #2008 review: the manifest still records gson, under the vendored uuid +/// or another one (the vendor entry, not the candidate's uuid, is what the +/// vendored basis attests). Before `sbt update` neither attests. +#[test] +fn sbt_vendored_vex_with_a_manifest_still_needs_evidence() { + let fx = Fx::vendored(); + for uuid in [UUID, OTHER_UUID] { + fx.restore_manifest(uuid); + let (_, env, doc) = fx.vex(); + assert!( + warning_codes(&env).contains(&"vex_sbt_resolution_unverified".to_string()), + "{uuid}: {env}" + ); + assert!(!attested(doc.as_ref()), "{uuid}: {doc:?}"); + } } #[test] diff --git a/crates/socket-patch-core/src/crawlers/sbt_evidence.rs b/crates/socket-patch-core/src/crawlers/sbt_evidence.rs index b6f6ffcc7..74d4faa15 100644 --- a/crates/socket-patch-core/src/crawlers/sbt_evidence.rs +++ b/crates/socket-patch-core/src/crawlers/sbt_evidence.rs @@ -665,3 +665,36 @@ pub(crate) fn stage_fixture(from: &Path, to: &Path) { #[cfg(test)] mod tests; + +/// Helpers for tests that stage sbt evidence on disk (this module's and +/// `vex::discover::sbt`'s). +#[cfg(test)] +pub(crate) mod testing { + use std::path::Path; + use std::time::SystemTime; + + /// Date `path` at `t`. + pub(crate) fn set_mtime(path: &Path, t: SystemTime) { + std::fs::OpenOptions::new() + .write(true) + .open(path) + .unwrap() + .set_modified(t) + .unwrap(); + } + + /// The `file:` URI sbt writes for `path`: `file:///C:/x` on Windows (a + /// backslash is no URI separator, and an invalid JSON escape). `path` + /// must not be a verbatim (`\\?\`) path, which sbt never records. + pub(crate) fn file_uri(path: &Path) -> String { + let text = path + .to_string_lossy() + .replace('\\', "/") + .replace(' ', "%20"); + if text.starts_with('/') { + format!("file://{text}") + } else { + format!("file:///{text}") + } + } +} diff --git a/crates/socket-patch-core/src/crawlers/sbt_evidence/tests.rs b/crates/socket-patch-core/src/crawlers/sbt_evidence/tests.rs index c09c3e762..b02bcef2d 100644 --- a/crates/socket-patch-core/src/crawlers/sbt_evidence/tests.rs +++ b/crates/socket-patch-core/src/crawlers/sbt_evidence/tests.rs @@ -5,6 +5,7 @@ use std::fs; use std::path::{Path, PathBuf}; use std::time::{Duration, SystemTime}; +use super::testing::{file_uri, set_mtime}; use super::*; const VERSIONS: &[&str] = &["0.13.18", "1.2.8", "1.3.13", "1.9.9", "1.13.0", "2.0.9"]; @@ -32,15 +33,6 @@ fn write(root: &Path, rel: &str, body: &[u8]) -> PathBuf { path } -fn set_mtime(path: &Path, t: SystemTime) { - fs::OpenOptions::new() - .write(true) - .open(path) - .unwrap() - .set_modified(t) - .unwrap(); -} - const JSON: &str = r#"{"configurations":[{"configuration":{"name":"compile"},"modules":[{"module":{"organization":"g","name":"a","revision":"1"},"artifacts":[],"evicted":false}],"details":[]}]}"#; fn rels(e: &SbtEvidence) -> Vec<&str> { @@ -358,20 +350,6 @@ fn symlinks_are_never_followed() { assert_eq!(e.build_sources.len(), 1); } -/// The `file:` URI sbt writes for `path`: `file:///C:/x` on Windows (a -/// backslash is no URI separator, and an invalid JSON escape). -fn file_uri(path: &Path) -> String { - let text = path - .to_string_lossy() - .replace('\\', "/") - .replace(' ', "%20"); - if text.starts_with('/') { - format!("file://{text}") - } else { - format!("file:///{text}") - } -} - #[test] fn cache_roots_from_artifact_paths() { let caches = tempfile::tempdir().unwrap(); diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 83fef2418..ffb12b571 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -606,6 +606,17 @@ pub enum UnattestedKind { /// version: the classifier-less pin does not retarget it and the grant /// serves only the main jar, so the build runs the public classifier jar. MavenClassifierUnpatched, + /// A vendored sbt pin (`socket-patch-vendor.sbt`) whose resolution the + /// build's local evidence does not verify, or under a build-wide + /// blocker (stale or partial evidence, a `dependencyOverrides` / + /// `resolvers` reassignment, a `build.sbt.lock`): the same conditions + /// that withhold a hosted sbt ref's lock pin. The vendor ledger, not a + /// ref, carries this wiring, so the mark stands alone. + SbtResolutionUnverified, + /// A vendored sbt pin whose committed `.socket/vendor/maven2` tree file + /// is missing or does not hash to the pinned sha256: sbt's load-time + /// check fails the build closed, so no copy is the patch. + SbtVendoredTreeMissing, /// A NuGet Socket source whose mapped id a governed `packages.lock.json` /// also locks at a version other than the patched one: the mapping /// routes every version of the id to a source that serves only the @@ -644,8 +655,9 @@ impl UnattestedKind { /// A ref discovery emits (so rollback, remove and list find the wiring, /// and the ledgers' liveness gates still see it wired) that must not be /// attested: the files show a build that may run a copy the pin does not -/// reach ([`UnattestedKind`]). The CLI's VEX plan omits every candidate of -/// `(purl, uuid)` with `detail`. Unlike a contest +/// reach ([`UnattestedKind`]). A vendored sbt pin is marked with no ref +/// beside it (its vendor ledger entry is the claim). The CLI's VEX plan +/// omits every candidate of `(purl, uuid)` with `detail`. Unlike a contest /// ([`Discovery::unpatched_copy`], [`Discovery::contest_across_locks`]), /// this never drops the ref, so `vendor --check` and `scan` keep treating /// the wiring as live — the right answer when re-running `scan` / `vendor` @@ -727,7 +739,10 @@ pub struct Discovery { /// [`Discovery::contest_within_locks`] ([`Discovery::unidentified_copy`]); /// always empty once discovery returns (folded into `unpatched_copies`). pub unidentified_copies: Vec, - /// Refs in `refs` whose wiring a build bypasses ([`Unattested`]). + /// Wired patches that must not be attested ([`Unattested`]): a ref in + /// `refs` whose wiring a build bypasses, or a vendored sbt pin whose + /// claim is its vendor ledger entry (no ref at all). Consumers match a + /// mark by `(purl, uuid)`, never by a ref. pub unattested: Vec, /// Copies no wiring reaches, pending [`Discovery::unattest_unwired_copies`]; /// always empty once discovery returns (folded into `unattested`). @@ -1397,8 +1412,10 @@ impl Discovery { }); } - /// Record that the ref `(purl, uuid)` is wired but bypassed by the - /// build `file` shows ([`Unattested`]). Pushed beside the ref itself. + /// Record that patch `uuid` is wired for `purl` but must not be + /// attested, for the reason `file` shows ([`Unattested`]). Pushed + /// beside the ref itself, or, for a vendored sbt pin (whose claim is + /// the vendor ledger's, not a ref's), alone. pub(crate) fn unattested( &mut self, purl: &str, diff --git a/crates/socket-patch-core/src/vex/discover/sbt.rs b/crates/socket-patch-core/src/vex/discover/sbt.rs index f0d166f89..3f2389eab 100644 --- a/crates/socket-patch-core/src/vex/discover/sbt.rs +++ b/crates/socket-patch-core/src/vex/discover/sbt.rs @@ -28,9 +28,15 @@ //! first checkout's `file:` origin (`docs/design/sbt-template-probe.md`, //! case i), and a copy whose bytes are the pinned ones is the patch. //! -//! Vendored pins are attested by the vendor ledger; this extractor only -//! diagnoses what would make that attestation a lie: a tree file gone or -//! changed (`vendored_tree_missing`) or an unverified resolution. +//! Vendored pins are attested by the vendor ledger; this extractor holds +//! them to the same evidence a hosted pin needs. A tree file gone or +//! changed (`vendored_tree_missing`) or a resolution the evidence does not +//! verify (`sbt_resolution_unverified`, the hosted check and build-wide +//! blockers verbatim) is diagnosed AND marked [`Unattested`] for the pin's +//! `(purl, uuid)`, so `vex` omits the ledger entry. Its liveness is +//! untouched: the wiring is intact, and `scan --prune` must not revert a +//! pin only because `sbt update` has not run since. A memory view has no +//! committed tree to hash, so it reports none missing. //! //! A generated file socket-patch cannot parse strictly is //! `sbt_owned_file_modified` (a foreign file under the name is not ours and @@ -307,7 +313,8 @@ async fn bypass(pin: &SbtPin, build: &BuildState, hashes: &mut Hashes) -> Option /// declared project with no evidence (only part of the build resolved), a /// project definition that cannot be read statically, a build source /// dropping the pin ([`pin_dropped`]) or reassigning `resolvers`, or a -/// `build.sbt.lock`. +/// `build.sbt.lock`. Each reason names its own remedy (or that there is +/// none), so a reader can relay it as is. fn evidence_blocker( e: &crate::crawlers::sbt_evidence::SbtEvidence, res: &JvmResolution, @@ -321,8 +328,10 @@ fn evidence_blocker( } let Some(declared) = declared_projects(&e.build_sources) else { return Some( - "the build's project definitions cannot be read statically, so its evidence may \ - not cover every project" + "the build's project definitions cannot be read statically (a computed \ + `Project(...)`, `projectMatrix` or `crossProject`), so its evidence may not cover \ + every project; only a build whose projects are literal `project` / \ + `Project(\"\", file(\"\"))` definitions can be verified" .to_string(), ); }; @@ -338,17 +347,25 @@ fn evidence_blocker( )); } if let Some(why) = pin_dropped(findings) { - return Some(why); + let remedy = if findings.overrides_assignment.is_empty() { + "keep its previous value (compose with `(Global / onLoad).value andThen ...`)" + } else { + "keep the setting's existing entries (append with `+=` / `++=`)" + }; + return Some(format!("{why}; {remedy}")); } if !findings.resolvers_assignment.is_empty() { return Some(format!( - "{} reassign(s) `resolvers`, which can drop the pin's repository", + "{} reassign(s) or transform(s) a resolver setting (`resolvers`, \ + `externalResolvers` or `fullResolvers`, with `:=` or `~=`), which can drop the \ + pin's repository; keep the setting's existing entries (append with `+=` / `++=`)", findings.resolvers_assignment.join(", ") )); } if !findings.dependency_lock.is_empty() { return Some(format!( - "{} locks the build's dependencies, which rejects the pinned version", + "{} locks the build's dependencies, which rejects the pinned version; the pin \ + cannot be verified while it exists", findings.dependency_lock.join(", ") )); } @@ -424,7 +441,9 @@ async fn verify( } if let Some(other) = versions.iter().find(|v| **v != pin.sv) { return Err(format!( - "the build resolves {ga}:{other}, not the pinned {}", + "the build resolves {ga}:{other}, not the pinned {}; run `sbt update`, and if it \ + still does, something in the build forces that version (a `dependencyOverrides \ + +=` or `force()` of it, or a project the pin does not reach): remove it", pin.sv )); } @@ -436,7 +455,8 @@ async fn verify( .collect(); if paths.is_empty() { return Err(format!( - "the build's resolution evidence records no artifact file for {ga}:{}", + "the build's resolution evidence records no artifact file for {ga}:{}, so it \ + cannot show which copy the build uses; run `sbt update`", pin.sv )); } @@ -447,14 +467,17 @@ async fn verify( Some(got) if got == pin.pom_sha256 => {} Some(got) => { return Err(format!( - "{ga}:{} resolved to {} with sha256 {got}, which is not pinned", + "{ga}:{} resolved to {} with sha256 {got}, which is not pinned (a copy \ + another repository serves, such as `mavenLocal` or an Ivy-local publish, \ + or a modified file): remove or restore it, then run `sbt update`", pin.sv, path.display() )) } None => { return Err(format!( - "{ga}:{} resolved to {}, which cannot be read", + "{ga}:{} resolved to {}, which cannot be read; restore it or run \ + `sbt update`", pin.sv, path.display() )) @@ -463,7 +486,7 @@ async fn verify( } if !jar_seen { return Err(format!( - "no resolved artifact of {ga}:{} is the pinned jar", + "no resolved artifact of {ga}:{} is the pinned jar; run `sbt update`", pin.sv )); } @@ -549,37 +572,65 @@ async fn vendored_check( hashes: &mut Hashes, out: &mut Discovery, ) { - for (ext, want) in [("pom", &pin.pom_sha256), ("jar", &pin.jar_sha256)] { - let rel = format!("{VENDORED_REPO_REL}/{}", pin.repo_path(ext)); - let got = match ctx.disk_root() { - Some(root) => hashes.of(&root.join(&rel)).await, - None => None, - }; - if got.as_deref() != Some(want.as_str()) { - out.diag( - DIAG_VENDORED_TREE_MISSING, - VENDORED_FILE, - format!( - "{VENDORED_FILE}: patch {} pins {rel}, which is {}", - pin.uuid, - if got.is_some() { "modified" } else { "missing" } - ), - ); - return; - } - } - if let Err(why) = verify_with(pin, res, blocker, hashes).await { + // The vendor ledger records the pin under this purl (the planner builds + // it the same way); a coordinate it rejects was never vendored. `why` + // is both the diagnostic's text and the mark's detail, which completes + // "patch is wired, but …" (the VEX note). + let purl = maven_purl(&pin.group, &pin.artifact, &pin.base); + let withhold = |out: &mut Discovery, code, why: String, kind| { out.diag( - DIAG_SBT_RESOLUTION_UNVERIFIED, + code, VENDORED_FILE, format!("{VENDORED_FILE}: patch {}: {why}", pin.uuid), ); + if let Some(purl) = &purl { + out.unattested(purl, &pin.uuid, VENDORED_FILE, why, kind); + } + }; + // Only a disk holds the committed tree: a memory view has none to + // hash, which proves nothing missing. + if let Some(root) = ctx.disk_root() { + for (ext, want) in [("pom", &pin.pom_sha256), ("jar", &pin.jar_sha256)] { + let rel = format!("{VENDORED_REPO_REL}/{}", pin.repo_path(ext)); + let got = hashes.of(&root.join(&rel)).await; + if got.as_deref() != Some(want.as_str()) { + let state = if got.is_some() { "modified" } else { "missing" }; + withhold( + out, + DIAG_VENDORED_TREE_MISSING, + format!("{VENDORED_FILE} pins {rel}, which is {state}"), + UnattestedKind::SbtVendoredTreeMissing, + ); + return; + } + } + } + // Without a disk there is no evidence either: unverified, exactly as a + // hosted pin read from memory gets no lock pin (and `sbt update` cannot + // change that, so the detail says why instead). + let verdict = if ctx.disk_root().is_none() { + Err( + "the project is read from memory, which holds no sbt resolution evidence; \ + only a scan of the checkout on disk can verify the pin" + .to_string(), + ) + } else { + verify_with(pin, res, blocker, hashes).await + }; + if let Err(why) = verdict { + withhold( + out, + DIAG_SBT_RESOLUTION_UNVERIFIED, + why, + UnattestedKind::SbtResolutionUnverified, + ); } } #[cfg(test)] mod tests { use super::*; + use crate::crawlers::sbt_evidence::testing::{file_uri, set_mtime}; use sha2::{Digest as _, Sha256}; const UUID: &str = "abcdef12-0000-4000-8000-000000000001"; @@ -738,6 +789,231 @@ mod tests { ); } + // ── vendored pins: the hosted evidence gate, end to end ──────────── + + const GSON_UUID: &str = "1d3c1fd2-5e6f-4a7b-8c9d-0e1f2a3b4c5d"; + const GSON_PURL: &str = "pkg:maven/com.google.code.gson/gson@2.8.9"; + const GSON_BUILD: &str = + "libraryDependencies += \"com.google.code.gson\" % \"gson\" % \"2.8.9\"\n"; + const EVIDENCE: &str = "target/update/update_cache/output"; + + fn later(secs: u64) -> std::time::SystemTime { + std::time::SystemTime::now() + std::time::Duration::from_secs(secs) + } + + fn earlier(secs: u64) -> std::time::SystemTime { + std::time::SystemTime::now() - std::time::Duration::from_secs(secs) + } + + /// An sbt 1.9.9 build with gson vendored for real (generated file, + /// committed tree, ledger) and, unless `evidence` is false, the record + /// `sbt update` writes after it: the suffixed version resolved from the + /// tree jar, newer than every build source. `(tmp, root, pin)`. + async fn vendored_gson(evidence: bool) -> (tempfile::TempDir, PathBuf, SbtPin) { + use crate::vendor::jvm::testing::{populate, vendor}; + use crate::vendor::jvm::{JvmPatch, Shape}; + // The raw tempdir path: a canonical Windows path is verbatim + // (`\\?\C:\…`), which sbt never records. + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path().to_path_buf(); + populate( + &root, + &[ + ("project/build.properties", "sbt.version=1.9.9\n"), + ("build.sbt", GSON_BUILD), + ], + ); + let patch = JvmPatch { + group_id: "com.google.code.gson", + artifact_id: "gson", + version: "2.8.9", + uuid: GSON_UUID, + jar: b"PK-gson", + upstream_pom: b"\n com.google.code.gson\n \ + gson\n 2.8.9\n\n", + upstream_module: None, + extra_artifacts: &[], + patched_members: &[], + }; + let mut ledger = BTreeMap::new(); + vendor(&root, Shape::Sbt, &patch, &mut ledger) + .await + .expect("vendor gson"); + let text = std::fs::read_to_string(root.join(VENDORED_FILE)).unwrap(); + let pin = parse(SbtFileMode::Vendored, &text) + .unwrap() + .pins + .remove(GSON_UUID) + .expect("gson pin"); + if evidence { + let jar = root.join(VENDORED_REPO_REL).join(pin.repo_path("jar")); + assert!(jar.is_file(), "{}", jar.display()); + let json = format!( + r#"{{"configurations":[{{"configuration":{{"name":"compile"}},"modules":[{{"module":{{"organization":"com.google.code.gson","name":"gson","revision":"{}"}},"artifacts":[[{{"name":"gson"}},"{}"]],"evicted":false}}],"details":[]}}]}}"#, + pin.sv, + file_uri(&jar) + ); + populate(&root, &[(EVIDENCE, json.as_str())]); + set_mtime(&root.join(EVIDENCE), later(60)); + } + (tmp, root, pin) + } + + /// The `(purl, uuid, kind)` of every [`super::super::Unattested`] mark. + fn marks(out: &Discovery) -> Vec<(String, String, UnattestedKind)> { + out.unattested + .iter() + .map(|u| (u.purl.clone(), u.uuid.clone(), u.kind)) + .collect() + } + + fn unverified() -> Vec<(String, String, UnattestedKind)> { + vec![( + GSON_PURL.to_string(), + GSON_UUID.to_string(), + UnattestedKind::SbtResolutionUnverified, + )] + } + + /// Fresh evidence resolving the pinned bytes from the tree: no + /// diagnostic, nothing withheld (the ledger attests it). + #[tokio::test] + async fn vendored_pin_with_verifying_evidence_is_attestable() { + let (_tmp, root, _) = vendored_gson(true).await; + let out = super::super::discover_patched_refs(&root).await; + assert!(out.diagnostics.is_empty(), "{:#?}", out.diagnostics); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + } + + /// The owner rule: a vendored sbt pin needs the evidence a hosted one + /// does, so with none (no `sbt update` since vendoring) the ledger + /// entry is withheld from VEX, not only diagnosed. + #[tokio::test] + async fn vendored_pin_without_evidence_is_unattested() { + let (_tmp, root, _) = vendored_gson(false).await; + let out = super::super::discover_patched_refs(&root).await; + assert_eq!(marks(&out), unverified(), "{:#?}", out.diagnostics); + assert!( + out.unattested[0] + .detail + .contains("no sbt resolution evidence"), + "{:?}", + out.unattested[0] + ); + } + + /// Evidence showing the pinned bytes, but older than a build source + /// edited since: it proves nothing about the build as it stands. + #[tokio::test] + async fn vendored_pin_with_stale_evidence_is_unattested() { + let (_tmp, root, _) = vendored_gson(true).await; + set_mtime(&root.join("build.sbt"), later(120)); + let out = super::super::discover_patched_refs(&root).await; + assert_eq!(marks(&out), unverified(), "{:#?}", out.diagnostics); + assert!( + out.unattested[0].detail.contains("changed after the last"), + "{:?}", + out.unattested[0] + ); + } + + /// A project reassigning `dependencyOverrides` replaces the generated + /// override where it applies, whatever the (fresh) evidence shows. + #[tokio::test] + async fn vendored_pin_under_an_overrides_reassignment_is_unattested() { + let (_tmp, root, _) = vendored_gson(true).await; + std::fs::write( + root.join("build.sbt"), + format!( + "{GSON_BUILD}dependencyOverrides := Seq(\"com.google.code.gson\" % \"gson\" % \ + \"2.8.9\")\n" + ), + ) + .unwrap(); + // Every build source still older than the evidence, however slow + // the run. + for rel in ["build.sbt", VENDORED_FILE, "project/build.properties"] { + set_mtime(&root.join(rel), earlier(60)); + } + let out = super::super::discover_patched_refs(&root).await; + assert_eq!(marks(&out), unverified(), "{:#?}", out.diagnostics); + assert!( + out.unattested[0].detail.contains("`dependencyOverrides`"), + "{:?}", + out.unattested[0] + ); + assert!( + out.diagnostics + .iter() + .any(|d| d.code == DIAG_SBT_RESOLUTION_UNVERIFIED), + "{:#?}", + out.diagnostics + ); + } + + /// A tree file gone: sbt's load-time check fails the build, so no + /// copy is the patch. + #[tokio::test] + async fn vendored_pin_with_a_missing_tree_file_is_unattested() { + let (_tmp, root, pin) = vendored_gson(true).await; + std::fs::remove_file(root.join(VENDORED_REPO_REL).join(pin.repo_path("pom"))).unwrap(); + let out = super::super::discover_patched_refs(&root).await; + assert_eq!( + marks(&out), + vec![( + GSON_PURL.to_string(), + GSON_UUID.to_string(), + UnattestedKind::SbtVendoredTreeMissing, + )], + "{:#?}", + out.diagnostics + ); + assert!( + out.unattested[0].detail.ends_with(".pom, which is missing") + && out.unattested[0].detail.starts_with(VENDORED_FILE), + "{:?}", + out.unattested[0] + ); + // The diagnostic and the mark say the same thing. + let diag = out + .diagnostics + .iter() + .find(|d| d.code == DIAG_VENDORED_TREE_MISSING) + .expect("diagnosed"); + assert!(diag.detail.ends_with(&out.unattested[0].detail), "{diag:?}"); + } + + /// A memory view holds no committed tree to hash: no tree is reported + /// missing (a gating mark discovery cannot back), while the pin is + /// still unverified, as a hosted pin read from memory gets no lock pin. + #[tokio::test] + async fn vendored_pin_over_a_memory_view_reports_no_missing_tree() { + let (_tmp, root, _) = vendored_gson(true).await; + let mut project = crate::vendor::lock_inventory::MemoryProject::new(); + for rel in [VENDORED_FILE, "build.sbt", "project/build.properties"] { + project.insert_text(rel, std::fs::read_to_string(root.join(rel)).unwrap()); + } + let out = super::super::discover_patched_refs_view( + crate::vendor::lock_inventory::ProjectView::Memory(&project), + &super::super::DiscoverOptions::default(), + ) + .await; + assert!( + !out.diagnostics + .iter() + .any(|d| d.code == DIAG_VENDORED_TREE_MISSING), + "{:#?}", + out.diagnostics + ); + assert_eq!(marks(&out), unverified(), "{:#?}", out.diagnostics); + // `sbt update` cannot fix a memory view: the detail says why. + assert!( + out.unattested[0].detail.contains("read from memory"), + "{:?}", + out.unattested[0] + ); + } + /// One record `rel` resolving lang3 at `v` (its id `rel:compile`), with /// `artifact` as the pinned version's file. fn record(r: &mut JvmResolution, rel: &str, v: &str, artifact: Option<&Path>) { @@ -880,15 +1156,6 @@ mod tests { std::fs::write(root.join(HOSTED_FILE), file).unwrap(); } - fn set_mtime(path: &Path, at: std::time::SystemTime) { - std::fs::File::options() - .write(true) - .open(path) - .unwrap() - .set_modified(at) - .unwrap(); - } - /// The build's own evidence: of the 1.9.9 test-compile build (root /// aggregating `a`, `b`, `c`; only `b` resolves gson 2.8.9), wired /// with a gson pin after `b` last resolved, and every other project diff --git a/docs/ecosystems.md b/docs/ecosystems.md index bd59b6321..c3e2610a1 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -1270,7 +1270,15 @@ the repository list, so it is left as a - The in-memory hosted engine (GitHub App) has no `target/` evidence, so it never wires sbt. - Vendored sbt VEX attests through the vendor ledger; the generated file alone - is not a VEX reference. + is not a VEX reference. The ledger entry needs the same `target/` evidence + as a hosted pin, and is omitted as `vex_sbt_resolution_unverified` while + that evidence does not show the build resolving the pinned bytes: before + `sbt update` has run since vendoring or since a build source changed, + while a build source reassigns `dependencyOverrides` / a resolver setting with + `:=`, `~=` or `--=`, or a `build.sbt.lock` exists, while something forces another + version, and always for a build whose projects cannot be read statically + (`projectMatrix`, `crossProject`, a computed `Project(...)`). The note + names the cause and its remedy. ## Cargo: shared registry cache