diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index bcbf0fa68..4a98987a4 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -647,7 +647,7 @@ to **six flavors**. | pypi / poetry (poetry.lock: legacy `[metadata.hashes]`, lock 1.0/1.1 `[metadata.files]`, 2.x `files`) | (rebuilt wheel) | lock-only: the target `[[package]]` gets `[package.source] type="file"` (+ `reference = ""` on the 0.12/1.0 layouts, which read it unconditionally) and the single `{file, hash: sha256-of-our-wheel}` entry in whichever table the generation keeps it. pyproject + `metadata.content-hash` untouched; CRLF locks keep their line endings. A lock written by Poetry < 1.4 emits `pypi_poetry_integrity_unverified` (that installer verifies no local hashes and skips an already-installed version) | `poetry check --lock && poetry sync`, cold cache (hash fail-closed from Poetry 1.4; byte-stable lock) — see `docs/testing/poetry-compatibility.md` | | pypi / pdm (pdm.lock) | (rebuilt wheel) | lock-only: the `[[package]]` gains the local-file `path` + `files[]` hash. pyproject + `content_hash` untouched. Non-fixture `[metadata] strategy` / hash-less locks refused | `pdm sync` (+ `pdm install --check`), cold cache | | pypi / pipenv (Pipfile.lock) | (rebuilt wheel) | lock-only: the `default`/`develop` entry → `{file, hashes:[sha256-of-our-wheel]}`. Pipfile + `_meta.hash` untouched. Emits `vendor_integrity_unverified` — pipenv does not hash-check file entries; the committed wheel bytes are the protection | `pipenv install --deploy` (+ `pipenv verify`), cold cache | -| pypi / requirements.txt (pip / `uv pip`) | (rebuilt wheel) | pin line → `./ --hash=sha256:` (markers carried over; transitive deps appended) | `pip install -r` / `uv pip install -r` **run from the project root** (both resolve bare paths against the CWD) | +| pypi / requirements.txt (pip / `uv pip`) | (rebuilt wheel) | pin line → `./` (markers carried over; transitive deps appended), plus `--hash=sha256:` only when the requirements tree is already in pip's hash-checking mode (any `--hash` or `--require-hashes`) | `pip install -r` / `uv pip install -r` **run from the project root** (both resolve bare paths against the CWD) | | nuget | deterministically rebuilt `.nupkg` at `..nupkg` (the uuid dir IS a NuGet folder feed; the stale embedded signature is dropped — unsigned is accepted under NuGet's default validation) | `nuget.config` source + `packageSourceMapping` for the id (creating the mapping from scratch ALSO fans a `` out to every pre-existing source — mapping is exclusive, NU1100 otherwise) **+** `packages.lock.json` `contentHash` → `base64(sha512(nupkg))` when the lock exists (`vendor_nuget_no_lockfile` warning otherwise) | `dotnet restore --locked-mode`, cold cache, `--network none` (tampered nupkg fails NU1403) | | maven | deterministically rebuilt `.jar` + the **verbatim upstream pom** (transitives survive; refused via `vendor_maven_pom_unavailable` rather than fabricated) + `.sha1` sidecars, laid out as a maven2 repository under the uuid dir | `pom.xml` `` (`id=socket-patch-vendor-`, `url=file://${project.basedir}/.socket/vendor/maven/`, `checksumPolicy=fail`, snapshots disabled). Multi-module aggregator poms refused (`vendor_maven_multimodule_unsupported`); gradle-only projects refused (`vendor_gradle_unsupported`); always-on `vendor_maven_local_cache_shadow` advisory (warm `~/.m2` wins over any repository) | `mvn` build on a fresh checkout with the GAV purged from the local repo, `--network none` (docker capstone; note `mvn -o` refuses `file://` repositories outright) | @@ -682,7 +682,7 @@ worse, lets a warm cache silently serve unpatched bytes): | pypi / poetry | `files = [{file, hash}]` (2.x) / `[metadata.files]` entry (1.0/1.1) / `[metadata.hashes]` entry (0.12) | replaced with a single `{file, hash: sha256-of-our-wheel}` (or the bare hash for 0.12) in the generation's own table (Poetry ≥ 1.4 verifies the artifact against one listed hash; older writers are flagged `pypi_poetry_integrity_unverified`; stale registry hashes removed) | | pypi / pdm | `[[package]].files[]` hashes | replaced with our wheel's sha256; hash-less locks refused (`pypi_pdm_lock_no_hashes`) | | pypi / pipenv | per-entry `hashes[]` | replaced with `["sha256:"]` — but pipenv does **not** enforce hashes on file entries (`vendor_integrity_unverified` warning); the committed wheel bytes are the actual protection | -| pypi / requirements | `--hash=sha256:` | fresh hash of the rebuilt wheel always emitted (turns on pip's hash-checking for the line) | +| pypi / requirements | `--hash=sha256:` | fresh hash of the rebuilt wheel emitted only when the requirements tree is already in pip's hash-checking mode; an unhashed tree stays unhashed, since one `--hash` turns the mode on for every requirement (#376) | ### Ownership, state, and reversal diff --git a/crates/socket-patch-cli/tests/e2e_hosted_production.rs b/crates/socket-patch-cli/tests/e2e_hosted_production.rs index 1ef761175..c18c0312e 100644 --- a/crates/socket-patch-cli/tests/e2e_hosted_production.rs +++ b/crates/socket-patch-cli/tests/e2e_hosted_production.rs @@ -1930,10 +1930,14 @@ fn pypi_requirements_txt_hosted_install_proof() { assert_redirected(&env_json, "requirements.txt"); let reqs = read(&proj.join("requirements.txt")); assert_hosted_pin(&reqs, PYPI_UUIDS, LEG); + // An unhashed file is pinned by the url's `#sha256=` fragment, which + // pip and uv both verify; a `--hash` would put pip in hash-checking + // mode for every other requirement (#376). assert!( - reqs.contains("--hash=sha256:"), - "{LEG}: rewritten requirements.txt carries no --hash pin, so pip/uv \ - would install the hosted wheel unverified:\n{reqs}" + reqs.contains(".whl#sha256=") && !reqs.contains("--hash"), + "{LEG}: rewritten requirements.txt must pin the hosted wheel by its \ + url fragment (and add no --hash), or pip/uv would install it \ + unverified:\n{reqs}" ); std::fs::remove_dir_all(&venv).expect("rm venv"); diff --git a/crates/socket-patch-cli/tests/e2e_vendor_pypi_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_pypi_build.rs index f613e6907..e91fcde29 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_pypi_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_pypi_build.rs @@ -1033,8 +1033,9 @@ fn pip_requirements_vendor_fresh_checkout_no_index_and_revert() { ); assert_vendored_applied(&parse_envelope(&stdout)); - // Artifact + the rewritten pin line (the exact spike-tested shape: - // `./ --hash=sha256: # socket-patch vendor: six==1.16.0`). + // Artifact + the rewritten pin line (the spike-tested shape: + // `./ # socket-patch vendor: six==1.16.0`; `--hash=sha256:` + // only in a file already in pip's hash-checking mode, #376). let wheel = vendored_wheel(&proj); let wheel_rel = format!( ".socket/vendor/pypi/{UUID}/{}", @@ -1052,8 +1053,9 @@ fn pip_requirements_vendor_fresh_checkout_no_index_and_revert() { "the path line must be ./-prefixed and project-relative: {vendor_line}" ); assert!( - vendor_line.contains("--hash=sha256:"), - "the path line must pin the wheel hash (hardens every install): {vendor_line}" + !requirements.contains("--hash"), + "an unhashed requirements.txt must stay unhashed, or pip's \ + hash-checking mode refuses every other requirement (#376):\n{requirements}" ); assert!( !requirements @@ -1201,7 +1203,12 @@ fn pip_vendored_requirements_evaluate_environment_markers() { "install upstream six", ); let patched = stage_patch(&project, &site_packages(&venv).join("six.py")); - let original = format!("six==1.16.0 ; {marker}\n"); + // Hash-pinned (pip-compile style), so the fresh install below can run + // `--require-hashes`: a hashed file keeps the vendor line hashed + // (#376), and the marker must survive next to the `--hash`. + let original = format!( + "six==1.16.0 ; {marker} \\\n --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254\n" + ); std::fs::write(project.join("requirements.txt"), &original).unwrap(); let (code, stdout, stderr) = run_vendored(&VendorDriver::VendorOffline, &project); assert_eq!(code, 0, "vendor failed: {stdout}\n{stderr}"); diff --git a/crates/socket-patch-cli/tests/e2e_vendored_production.rs b/crates/socket-patch-cli/tests/e2e_vendored_production.rs index 25bb2b36a..efe0f8daa 100644 --- a/crates/socket-patch-cli/tests/e2e_vendored_production.rs +++ b/crates/socket-patch-cli/tests/e2e_vendored_production.rs @@ -1891,8 +1891,9 @@ fn pypi_requirements_txt_vendored_install_proof() { "{LEG}: requirements.txt was not rewired to the vendored wheel:\n{reqs}" ); assert!( - reqs.contains("--hash=sha256:"), - "{LEG}: rewritten requirements.txt carries no --hash pin:\n{reqs}" + !reqs.contains("--hash"), + "{LEG}: an unhashed requirements.txt must stay unhashed, or pip's \ + hash-checking mode refuses every other requirement (#376):\n{reqs}" ); // DELIVERY PROOF: requirements.txt + .socket only, fresh venv, --no-index diff --git a/crates/socket-patch-cli/tests/e2e_vex_build/pip.rs b/crates/socket-patch-cli/tests/e2e_vex_build/pip.rs index 5a6d336c7..2adad6956 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_build/pip.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_build/pip.rs @@ -1,12 +1,13 @@ //! Real-pip capstone for manifest-less VEX over `requirements.txt`: for //! every pip major (latest release of each, `SOCKET_PATCH_PIP_E2E_VERSIONS` -//! overrides), HOSTED and VENDORED, three project shapes: +//! overrides), HOSTED and VENDORED, four project shapes: //! //! | cell | requirements | hosted | vendored | //! | --- | --- | --- | --- | //! | `root` | `six==1.16.0` | yes | yes | //! | `hashes` | `pip-compile --generate-hashes` style (`\` continued `--hash`, hash-checking mode) | yes | yes | //! | `include` | `-r requirements/base.txt` | root-only rewriter: stays on the registry, nothing attested | yes | +//! | `unhashed` | `six==1.16.0` + `idna==3.7`, no hashes: the wiring must not add a `--hash` (#376) | yes | yes | //! //! Each flow: //! @@ -57,6 +58,10 @@ enum Cell { Root, Hashes, Include, + /// #376: a second, unhashed requirement next to the patched one. One + /// `--hash` on the wired line would put pip in hash-checking mode for + /// the whole install and refuse `idna==3.7`. + Unhashed, } impl Cell { @@ -65,6 +70,7 @@ impl Cell { Cell::Root => "root", Cell::Hashes => "hashes", Cell::Include => "include", + Cell::Unhashed => "unhashed", } } @@ -82,6 +88,7 @@ impl Cell { ("requirements.txt", "-r requirements/base.txt\n".into()), ("requirements/base.txt", "six==1.16.0\n".into()), ], + Cell::Unhashed => vec![("requirements.txt", "six==1.16.0\nidna==3.7\n".into())], } } } @@ -221,10 +228,19 @@ fn flow(uv: &Path, major: &str, pip_version: &str, cell: Cell, mode: Mode, root: record("pip", pip_version, &row, "embedded-scan-vex", "pass"); std::fs::remove_file(&embedded).unwrap(); let wired = wiring_text(&proj, cell); + // pip's hash-checking mode is all or nothing (#376): only an already + // hashed file gets a `--hash`; the hosted url otherwise carries the + // pin as a `#sha256=` fragment pip verifies without the mode. + assert_eq!( + wired.contains("--hash="), + cell == Cell::Hashes, + "{what}: the wiring must keep the file's hash-checking mode: {wired}" + ); match mode { Mode::Hosted => assert!( - wired.contains(&api.artifact_url()) && wired.contains("--hash=sha256:"), - "{what}: requirements must point at the hosted wheel: {wired}" + wired.contains(&api.artifact_url()) + && (cell == Cell::Hashes || wired.contains(".whl#sha256=")), + "{what}: requirements must point at the pinned hosted wheel: {wired}" ), Mode::Vendored => assert!( wired.contains(&format!(".socket/vendor/pypi/{}/", mode.uuid())), @@ -239,11 +255,14 @@ fn flow(uv: &Path, major: &str, pip_version: &str, cell: Cell, mode: Mode, root: let fresh_venv = fresh.join(".venv"); pip_venv(uv, major, &fresh_venv).unwrap_or_else(|e| panic!("{what}: fresh venv: {e}")); let downloads = api.artifact_downloads(); - let out = pip( - &fresh_venv, - &fresh, - &["install", "--no-index", "-r", "requirements.txt"], - ); + // The unhashed cell's other requirement (idna) comes from PyPI; the + // patched six still can only come from the wiring. + let install: &[&str] = if cell == Cell::Unhashed { + &["install", "-r", "requirements.txt"] + } else { + &["install", "--no-index", "-r", "requirements.txt"] + }; + let out = pip(&fresh_venv, &fresh, install); assert_ok(&out, &format!("{what}: fresh `pip install --no-index -r`")); let (_, bytes, is_patched) = six_oracle(&venv_bin(&fresh_venv, "python"), &fresh) .unwrap_or_else(|| panic!("{what}: six not importable in the fresh checkout")); @@ -335,7 +354,7 @@ fn pip_every_major_hosted_and_vendored_end_in_manifest_less_vex() { continue; } }; - for cell in [Cell::Root, Cell::Hashes, Cell::Include] { + for cell in [Cell::Root, Cell::Hashes, Cell::Include, Cell::Unhashed] { for mode in [Mode::Hosted, Mode::Vendored] { let root = scratch.path().join("run"); let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { diff --git a/crates/socket-patch-cli/tests/in_process_get_hosted_ecosystems.rs b/crates/socket-patch-cli/tests/in_process_get_hosted_ecosystems.rs index ff45dcd4b..6185b923c 100644 --- a/crates/socket-patch-cli/tests/in_process_get_hosted_ecosystems.rs +++ b/crates/socket-patch-cli/tests/in_process_get_hosted_ecosystems.rs @@ -174,9 +174,11 @@ fn assert_no_manifest_no_blobs(cwd: &Path) { // --------------------------------------------------------------------------- /// A pip project pinning `requests==2.31.0`: the hosted grant must rewrite -/// that one line to `requests @ --hash=sha256:` (the -/// integrity pin fails closed on tampered bytes), leave the bystander line -/// byte-identical — and write no manifest and no ledger. +/// that one line to `requests @ #sha256=` (the integrity +/// pin fails closed on tampered bytes; a url fragment, not `--hash`, since +/// one `--hash` would put pip in hash-checking mode for the unhashed +/// `flask` line too — #376), leave the bystander line byte-identical — and +/// write no manifest and no ledger. #[tokio::test] #[serial] async fn pypi_requirements_hosted_rewrites_pinned_line() { @@ -211,7 +213,7 @@ async fn pypi_requirements_hosted_rewrites_pinned_line() { assert_eq!(code, 0, "get --mode hosted (pypi) should succeed"); let reqs = std::fs::read_to_string(tmp.path().join("requirements.txt")).unwrap(); - let expected_line = format!("requests @ {url} --hash=sha256:{SHA256}"); + let expected_line = format!("requests @ {url}#sha256={SHA256}"); assert!( reqs.lines().any(|l| l == expected_line), "requirements.txt must pin the hosted wheel URL + sha256; got:\n{reqs}" @@ -233,7 +235,7 @@ async fn pypi_requirements_hosted_rewrites_pinned_line() { // Manifest-less VEX over what `get --mode hosted` committed (it never // writes a manifest). An EMPTY in-project venv keeps the crawl hermetic - // (nothing installed: the `--hash` pin is the evidence); the grant's + // (nothing installed: the `#sha256=` pin is the evidence); the grant's // origin is this test's patch server, hence `--patch-server-url`. let site = if cfg!(windows) { tmp.path().join(".venv/Lib/site-packages") diff --git a/crates/socket-patch-cli/tests/vendor_ledger_schema_e2e.rs b/crates/socket-patch-cli/tests/vendor_ledger_schema_e2e.rs index 819e7c1c1..94155fde7 100644 --- a/crates/socket-patch-cli/tests/vendor_ledger_schema_e2e.rs +++ b/crates/socket-patch-cli/tests/vendor_ledger_schema_e2e.rs @@ -208,6 +208,25 @@ fn new_ledgers_compact_whole_file_snapshots_and_revert() { } } +/// `bytes` with every ` --hash=sha256:<64 hex>` option removed. +fn strip_sha256_hash_options(bytes: &[u8]) -> Vec { + const NEEDLE: &[u8] = b" --hash=sha256:"; + let mut out = Vec::with_capacity(bytes.len()); + let mut i = 0; + while i < bytes.len() { + let hex = bytes.get(i + NEEDLE.len()..i + NEEDLE.len() + 64); + if bytes[i..].starts_with(NEEDLE) + && hex.is_some_and(|h| h.iter().all(u8::is_ascii_hexdigit)) + { + i += NEEDLE.len() + 64; + } else { + out.push(bytes[i]); + i += 1; + } + } + out +} + /// Against the integrated base: for every ecosystem this binary wires /// exactly the files the base binary wired (the checked-in legacy /// fixtures), and its ledger — whatever its on-disk version — loads to the @@ -215,7 +234,17 @@ fn new_ledgers_compact_whole_file_snapshots_and_revert() { #[tokio::test] async fn server_artifacts_preserve_legacy_wiring_shape_and_originals() { for eco in fx::ALL { - let base_wired = read_tree(&fixtures_dir().join(eco).join("wired")); + let mut base_wired = read_tree(&fixtures_dir().join(eco).join("wired")); + if *eco == "pypi-requirements" { + // The one intended difference: the base binary pinned its vendor + // lines with `--hash` even in this unhashed requirements.txt, + // which put pip in hash-checking mode for every other + // requirement (#376). This binary writes the same lines without + // it — in the file and in the ledger's recorded `new` text. + for (_, bytes) in &mut base_wired { + *bytes = strip_sha256_hash_options(bytes); + } + } let f = Fixture::new(eco); let (code, stdout, stderr) = f.vendor(&[], &[]); assert_eq!(code, 0, "{eco}: {stdout}\n{stderr}"); diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 59d148cdf..753bfc4b1 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -6286,14 +6286,16 @@ mod tests { )]; let first = rewrite_registry_redirect(&files, &overrides); let out = first.files.get("requirements.txt").expect("rewritten"); + // An unhashed file pins by the url fragment (#376), so the marker + // follows it. assert_eq!( - out.matches("--hash=sha256:").count(), + out.matches("sha256").count(), 1, "exactly one hash after the first pass: {out}" ); assert!( - out.contains("; python_version >= \"3.7\" --hash="), - "marker preserved ahead of the hash: {out}" + out.contains("-none-any.whl#sha256=") && out.contains(" ; python_version >= \"3.7\"\n"), + "marker preserved after the pinned url: {out}" ); let mut again = files.clone(); @@ -6307,10 +6309,11 @@ mod tests { ); } - /// An inline comment after the marker must not swallow the appended - /// `--hash=…` (pip would then treat the hash as comment text and skip - /// enforcement). The comment is split off and re-appended AFTER the hash - /// so the pin stays active and the user's note survives. + /// An inline comment after the marker must not swallow the appended pin + /// (pip would then treat it as comment text and skip enforcement). The + /// comment is split off and re-appended AFTER the pin — the url's + /// `#sha256=` fragment in an unhashed file (#376), `--hash` in a hashed + /// one — so the pin stays active and the user's note survives. #[test] fn requirements_marker_comment_keeps_hash_active() { let original = "requests==2.28.1 ; python_version >= \"3.7\" # explanation\n"; @@ -6323,13 +6326,23 @@ mod tests { assert_eq!( output, &format!( - "requests @ {url} ; python_version >= \"3.7\" --hash=sha256:{sha256} # explanation\n" + "requests @ {url}#sha256={sha256} ; python_version >= \"3.7\" # explanation\n" ) ); let again = BTreeMap::from([("requirements.txt".to_string(), output.clone())]); let second = rewrite_registry_redirect(&again, &overrides); assert!(second.files.is_empty()); assert!(second.edits.is_empty()); + + let hashed = original.replace("# explanation", "--hash=sha256:old # explanation"); + let files = BTreeMap::from([("requirements.txt".to_string(), hashed)]); + let first = rewrite_registry_redirect(&files, &overrides); + assert_eq!( + first.files["requirements.txt"], + format!( + "requests @ {url} ; python_version >= \"3.7\" --hash=sha256:{sha256} # explanation\n" + ) + ); } const MAVEN_SUFFIXED: &str = "1.7.36-socket.aaaaaaaa"; diff --git a/crates/socket-patch-core/src/patch/redirect/requirements.rs b/crates/socket-patch-core/src/patch/redirect/requirements.rs index cdbd9eb6d..75816107f 100644 --- a/crates/socket-patch-core/src/patch/redirect/requirements.rs +++ b/crates/socket-patch-core/src/patch/redirect/requirements.rs @@ -207,6 +207,10 @@ pub(super) fn rewrite( .or_default() .insert(&dep.version); } + // pip's hash-checking mode is all or nothing (#376): pin the patched + // artifact with `--hash` only when the file already carries hashes + // (the replaced pin's own included), else by the url fragment. + let hashed = crate::utils::requirements::requires_hashes(content); let mut changed = false; for dep in overrides.iter().filter(|dep| dep.ecosystem == "pypi") { let Some(sha256) = &dep.integrity.sha256 else { @@ -265,7 +269,9 @@ pub(super) fn rewrite( } } matched = true; - result.confirmed_requirements_uuids.insert(dep.patch_uuid.clone()); + result + .confirmed_requirements_uuids + .insert(dep.patch_uuid.clone()); let options = requirement_tokens(specifier) .into_iter() .skip_while(|token| !token.starts_with("--")) @@ -285,14 +291,28 @@ pub(super) fn rewrite( } else { "" }; - let mut rewritten = format!("{bom}{indent}{}{extras} @ {}", dep.name, dep.artifact_url); + // Unhashed file: the url's `#sha256=` fragment, which pip + // verifies without turning hash-checking mode on. + let location = if hashed { + dep.artifact_url.clone() + } else { + let separator = if dep.artifact_url.contains('#') { + '&' + } else { + '#' + }; + format!("{}{separator}sha256={sha256}", dep.artifact_url) + }; + let mut rewritten = format!("{bom}{indent}{}{extras} @ {location}", dep.name); for suffix in [marker.trim(), options.as_str()] { if !suffix.is_empty() { rewritten.push(' '); rewritten.push_str(suffix); } } - rewritten.push_str(&format!(" --hash=sha256:{sha256}")); + if hashed { + rewritten.push_str(&format!(" --hash=sha256:{sha256}")); + } if !comment.is_empty() { rewritten.push(' '); rewritten.push_str(comment); @@ -426,7 +446,7 @@ mod tests { let result = rewrite_registry_redirect(&input(&source), &[patch()]); assert_eq!( result.files["requirements.txt"], - format!("{prefix}# documentation \\\nrequests @ {URL} --hash=sha256:{HASH}\n") + format!("{prefix}# documentation \\\nrequests @ {URL}#sha256={HASH}\n") ); } } @@ -485,7 +505,7 @@ mod tests { other.artifact_url = URL.replace("2.28.1", "2.32.0"); other.integrity.sha256 = Some("d".repeat(64)); let expected = format!( - "requests @ {URL} ; python_version < '3.10' --hash=sha256:{HASH}\nrequests @ {} ; python_version >= '3.10' --hash=sha256:{}\n", + "requests @ {URL}#sha256={HASH} ; python_version < '3.10'\nrequests @ {}#sha256={} ; python_version >= '3.10'\n", other.artifact_url, "d".repeat(64) ); @@ -524,9 +544,53 @@ mod tests { let result = rewrite_registry_redirect(&input(source), &[patch()]); assert_eq!( result.files["requirements.txt"], - format!("requests @ {URL} --hash=sha256:{HASH}") + format!("requests @ {URL}#sha256={HASH}") + ); + } + } + + /// #376: an unhashed requirements file must stay unhashed. pip turns + /// hash-checking mode on for the WHOLE install as soon as one line has a + /// `--hash`, so pinning only the patched line breaks every other + /// requirement (and every transitive dependency). The patched sha256 + /// rides in the url's `#sha256=` fragment instead, which pip verifies + /// without turning the mode on. + #[test] + fn unhashed_file_pins_the_artifact_by_url_fragment_not_hash_option() { + let source = "requests==2.28.1\nidna==3.7\n"; + let result = rewrite_registry_redirect(&input(source), &[patch()]); + let expected = format!("requests @ {URL}#sha256={HASH}\nidna==3.7\n"); + assert_eq!(result.files["requirements.txt"], expected); + assert!(!result.files["requirements.txt"].contains("--hash")); + assert!(result.warnings.is_empty(), "{:?}", result.warnings); + // Idempotent: the rewritten line keeps the file unhashed. + let rerun = rewrite_registry_redirect(&input(&expected), &[patch()]); + assert!(rerun.files.is_empty() && rerun.edits.is_empty()); + } + + /// #376: a file the user already hashes keeps `--hash` on the patched + /// line (hash-checking mode is on either way), whether the hashes sit on + /// another requirement or the file sets `--require-hashes`. + #[test] + fn hashed_file_keeps_the_hash_option() { + for other in [ + "idna==3.7 --hash=sha256:aaaa\n", + "idna==3.7 \\\n --hash sha512:bbbb\n", + "--require-hashes\nidna==3.7\n", + ] { + let source = format!("requests==2.28.1\n{other}"); + let result = rewrite_registry_redirect(&input(&source), &[patch()]); + assert_eq!( + result.files["requirements.txt"], + format!("requests @ {URL} --hash=sha256:{HASH}\n{other}"), + "{other:?}" ); } + // A hash in a comment, or a url fragment, is not a hash option. + let source = "requests==2.28.1\n# idna==3.7 --hash=sha256:aaaa\nsix @ https://files.pythonhosted.org/six-1.16.0-py2.py3-none-any.whl#sha256=dd\n"; + let result = rewrite_registry_redirect(&input(source), &[patch()]); + assert!(result.files["requirements.txt"] + .starts_with(&format!("requests @ {URL}#sha256={HASH}\n"))); } #[test] @@ -548,7 +612,7 @@ mod tests { let result = rewrite_registry_redirect(&input("requests\n"), &[patch()]); assert_eq!( result.files["requirements.txt"], - format!("requests @ {URL} --hash=sha256:{HASH}\n") + format!("requests @ {URL}#sha256={HASH}\n") ); let mut other = patch(); other.version = "2.32.0".into(); diff --git a/crates/socket-patch-core/src/utils/requirements.rs b/crates/socket-patch-core/src/utils/requirements.rs index f811032a6..5d3087e9b 100644 --- a/crates/socket-patch-core/src/utils/requirements.rs +++ b/crates/socket-patch-core/src/utils/requirements.rs @@ -157,6 +157,25 @@ pub(crate) fn hash_options(code: &str) -> Vec { hashes } +/// Whether a requirements file puts pip into hash-checking mode for the whole +/// install: pip turns it on as soon as ANY requirement carries a `--hash` +/// option (of any algorithm), or the file sets `--require-hashes`. The mode +/// is all or nothing: once on, every requirement — and every transitive +/// dependency — must be `==`-pinned and hashed, so a writer must match it +/// rather than add the first `--hash` (#376) or an unhashed line (#378). +/// +/// Every line counts, socket-patch's own included: a line this writer +/// emitted keeps the mode it was written for, so a re-scan is a no-op. A +/// url's `#sha256=` fragment is not a hash option: pip verifies it without +/// turning the mode on. +pub(crate) fn requires_hashes(content: &str) -> bool { + logical_lines(content).iter().any(|line| { + strip_comment(&line.text).split_whitespace().any(|token| { + token == "--hash" || token.starts_with("--hash=") || token == "--require-hashes" + }) + }) +} + /// `(distribution, version)` a Python artifact filename names: a PEP 427 /// wheel (`dist-version-…-tags.whl`) or an sdist (`dist-version.tar.gz` / /// `.zip` / `.tar.bz2` / `.tar.xz`). Names are returned as spelled (callers @@ -193,6 +212,28 @@ pub(crate) fn url_sha256_fragment(location: &str) -> Option { mod tests { use super::*; + #[test] + fn requires_hashes_reads_pip_hash_checking_mode() { + for hashed in [ + "six==1.16.0 --hash=sha256:aa\nidna==3.7\n", + "six==1.16.0 \\\n --hash sha256:aa\n", + "six==1.16.0 --hash=sha512:aa\n", + "--require-hashes\nsix==1.16.0\n", + "\u{feff}--require-hashes\r\nsix==1.16.0\r\n", + ] { + assert!(requires_hashes(hashed), "{hashed:?}"); + } + for unhashed in [ + "", + "six==1.16.0\nidna==3.7\n", + // Comments and url fragments are not hash options. + "six==1.16.0 # --hash=sha256:aa\n# --require-hashes\n", + "six @ https://example.test/six-1.16.0-py2.py3-none-any.whl#sha256=aa\n", + ] { + assert!(!requires_hashes(unhashed), "{unhashed:?}"); + } + } + #[test] fn lexer_joins_continuations_and_strips_comments_correctly() { let lines = logical_lines("six==1.16.0 \\\n --hash=sha256:abc\nrequests\n"); diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs index 6fa1f698b..3b793fa9c 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs @@ -2911,7 +2911,7 @@ async fn the_vendored_requirements_writers_own_output_reinventories() { let tmp = tempfile::tempdir().unwrap(); let line = crate::vendor::pypi_requirements::vendor_line( &format!(".socket/vendor/pypi/{UUID}/requests-2.28.1-py3-none-any.whl"), - &"c".repeat(64), + Some(&"c".repeat(64)), "requests", "2.28.1", &None, diff --git a/crates/socket-patch-core/src/vendor/pypi.rs b/crates/socket-patch-core/src/vendor/pypi.rs index ed5592dde..f557cc65e 100644 --- a/crates/socket-patch-core/src/vendor/pypi.rs +++ b/crates/socket-patch-core/src/vendor/pypi.rs @@ -985,6 +985,13 @@ async fn pypi_prelude<'p>( }) } +/// Whether a rebuilt wheel reproduces the in-sync pin. An empty pinned +/// sha256 comes from an unhashed requirements vendor line, which pins the +/// wheel path alone. +fn pin_matches(pin_path: &str, pin_sha: &str, rel_wheel: &str, sha256_hex: &str) -> bool { + pin_path == rel_wheel && (pin_sha.is_empty() || pin_sha == sha256_hex) +} + /// Whether [`vendor_pypi_with_pipenv_version`] — a wet run with the service /// enabled — asks the patch service for `record`: past every refusal it /// raises first, and answered neither by the in-sync hot path nor by the @@ -1153,7 +1160,7 @@ pub async fn vendor_pypi_with_pipenv_version<'a>( // `uv sync`, …) the moment vendor reports success. Sweep the // mismatched wheel back out and fail loudly instead. if let Some((pin_path, pin_sha)) = &expected_pin { - if *pin_path != rel_wheel || *pin_sha != artifact.sha256_hex { + if !pin_matches(pin_path, pin_sha, &rel_wheel, &artifact.sha256_hex) { let _ = tokio::fs::remove_dir_all(project_root.join(&uuid_dir_rel)).await; prune_empty_vendor_levels(&project_root.join(&uuid_dir_rel)).await; let mut result = result; @@ -1843,7 +1850,7 @@ async fn try_pypi_service_wheel( // Digested on first ask: pypi is the only backend that pins it. let sha256_hex = archive.sha256_hex().to_string(); if let Some((pin_path, pin_sha)) = expected_pin { - if *pin_path != rel_wheel || *pin_sha != sha256_hex { + if !pin_matches(pin_path, pin_sha, &rel_wheel, &sha256_hex) { return policy.miss( warnings, "vendor_prebuilt_pin_mismatch", @@ -2184,6 +2191,20 @@ mod tests { record: PatchRecord, } + /// [`e2e_fixture`] with a hash-pinned requirements.txt: pip's + /// hash-checking mode is on, so the vendor line carries the `--hash` + /// pin the in-sync rebuild guard reads back (#376). + async fn e2e_fixture_hashed() -> E2eFixture { + let fx = e2e_fixture().await; + touch( + &fx.root, + "requirements.txt", + &format!("six==1.16.0 --hash=sha256:{}\n", "0".repeat(64)), + ) + .await; + fx + } + /// A requirements-flavor project: requirements.txt at the root, a /// six-like install in a venv-ish site-packages, and a blob store. async fn e2e_fixture() -> E2eFixture { @@ -2291,16 +2312,15 @@ mod tests { hex::encode(sha2::Sha256::digest(&wheel_bytes)) ); - // The requirements line was rewritten with that exact hash. + // The requirements line was rewritten to the wheel path. The file + // had no hashes, so neither does the line (#376): one `--hash` + // would put pip in hash-checking mode for every requirement. let req = tokio::fs::read_to_string(fx.root.join("requirements.txt")) .await .unwrap(); assert_eq!( req, - format!( - "./{wheel_rel} --hash=sha256:{} # socket-patch vendor: six==1.16.0\n", - entry.artifact.sha256 - ) + format!("./{wheel_rel} # socket-patch vendor: six==1.16.0\n") ); assert_eq!(entry.wiring.len(), 1); assert_eq!(entry.wiring[0].kind, "requirements_line"); @@ -2927,7 +2947,7 @@ wheels = [ /// `vendor_prebuilt_downloaded` advisory is emitted. #[tokio::test] async fn service_success_requirements_writes_wheel_and_wires_sha256() { - let fx = e2e_fixture().await; + let fx = e2e_fixture_hashed().await; let sources = PatchSources::blobs_only(&fx.blobs); let bytes: &[u8] = &served_wheel(b"prebuilt wheel bytes from the service"); let sri = sri_sha512(bytes); @@ -3358,7 +3378,7 @@ wheels = [ /// build that reproduces the pin, exactly as with the ledger present. #[tokio::test] async fn in_sync_ledgerless_service_rebuild_must_not_break_wired_pin() { - let fx = e2e_fixture().await; + let fx = e2e_fixture_hashed().await; let sources = PatchSources::blobs_only(&fx.blobs); let VendorOutcome::Done { result, entry, .. } = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", @@ -3421,13 +3441,87 @@ wheels = [ ); } + /// An unhashed requirements set gets a hashless vendor line, which still + /// pins the wheel PATH. With no ledger entry, a service rebuild that + /// lands at another filename would leave that line pointing at nothing, + /// so the guard refuses it; different bytes at the pinned path break no + /// hash and are accepted. + #[tokio::test] + async fn in_sync_ledgerless_rebuild_of_unhashed_line_keeps_the_wired_path() { + let fx = e2e_fixture().await; + let sources = PatchSources::blobs_only(&fx.blobs); + let vendor = |cfg: Option| { + let (fx, sources) = (&fx, &sources); + async move { + crate::vendor::test_support::vendor_pypi( + "pkg:pypi/six@1.16.0", + &fx.site_packages, + &fx.root, + &fx.record, + sources, + "2026-06-09T00:00:00Z", + false, + false, + cfg.as_ref(), + ) + .await + } + }; + let VendorOutcome::Done { result, .. } = vendor(None).await else { + panic!("first vendor must be Done"); + }; + assert!(result.success, "{:?}", result.error); + let wired = tokio::fs::read_to_string(fx.root.join("requirements.txt")) + .await + .unwrap(); + assert!(!wired.contains("--hash"), "{wired}"); + let uuid_dir = fx.root.join(format!(".socket/vendor/pypi/{UUID}")); + + // Another filename: refused, requirements.txt untouched. + tokio::fs::remove_dir_all(&uuid_dir).await.unwrap(); + let bytes = served_wheel(b"service wheel at another filename"); + let server = wiremock::MockServer::start().await; + mount_pypi_granted(&server, "six-1.16.0-py3-none-any.whl", &sri_sha512(&bytes), &bytes) + .await; + let cfg = pypi_service_cfg(&server.uri(), VendorSource::Service, false); + let error = crate::vendor::test_support::expect_failure(vendor(Some(cfg)).await); + assert!( + error.contains("does not match the wheel the lockfile still pins"), + "{error}" + ); + assert_eq!( + tokio::fs::read_to_string(fx.root.join("requirements.txt")) + .await + .unwrap(), + wired + ); + + // Same filename, different bytes: rebuilt, requirements.txt untouched. + let _ = tokio::fs::remove_dir_all(&uuid_dir).await; + let bytes = served_wheel(b"service wheel at the pinned filename"); + let server = wiremock::MockServer::start().await; + mount_pypi_granted(&server, WHEEL_NAME, &sri_sha512(&bytes), &bytes).await; + let cfg = pypi_service_cfg(&server.uri(), VendorSource::Service, false); + let VendorOutcome::Done { result, .. } = vendor(Some(cfg)).await else { + panic!("same-path rebuild must be Done"); + }; + assert!(result.success, "{:?}", result.error); + assert!(uuid_dir.join(WHEEL_NAME).is_file()); + assert_eq!( + tokio::fs::read_to_string(fx.root.join("requirements.txt")) + .await + .unwrap(), + wired + ); + } + /// The ledgerless twin of the loud local failure: a project vendored /// FROM THE SERVICE whose ledger entry AND wheel are gone must not /// "rebuild" locally into bytes the wired requirements line does not /// pin — the wired file itself carries the pin the guard checks. #[tokio::test] async fn in_sync_ledgerless_local_rebuild_pin_mismatch_fails_loudly() { - let fx = e2e_fixture().await; + let fx = e2e_fixture_hashed().await; let sources = PatchSources::blobs_only(&fx.blobs); let bytes: &[u8] = &served_wheel(b"prebuilt wheel bytes from the service"); let sri = sri_sha512(bytes); @@ -5972,7 +6066,7 @@ wheels = [ #[tokio::test] async fn in_sync_rebuild_with_corrupt_ledger_falls_back_to_wired_pin() { - let fx = e2e_fixture().await; + let fx = e2e_fixture_hashed().await; let sources = PatchSources::blobs_only(&fx.blobs); let VendorOutcome::Done { result, entry, .. } = vendor_six(&fx, &sources, None).await else { diff --git a/crates/socket-patch-core/src/vendor/pypi_requirements.rs b/crates/socket-patch-core/src/vendor/pypi_requirements.rs index 2c531d17f..236e25dc0 100644 --- a/crates/socket-patch-core/src/vendor/pypi_requirements.rs +++ b/crates/socket-patch-core/src/vendor/pypi_requirements.rs @@ -1,13 +1,20 @@ //! requirements.txt wiring (pip & `uv pip`). //! //! The spike-verified line shape is -//! `./[ ; ] --hash=sha256: # socket-patch vendor: ==`: +//! `./[ ; ] [--hash=sha256:] # socket-patch vendor: ==`: //! both pip 26 and uv 0.11 accept the bare relative path (resolved against //! the INVOKING CWD, never the requirements-file dir — hence the documented -//! root-only constraint), enforce the `--hash` pin (implicitly: any -//! `--hash` on any line turns hash-checking on), strip the trailing comment, -//! and genuinely EVALUATE a `; marker` on a path line — so an environment -//! marker is carried over from the replaced pin instead of refused. +//! root-only constraint), enforce the `--hash` pin, strip the trailing +//! comment, and genuinely EVALUATE a `; marker` on a path line — so an +//! environment marker is carried over from the replaced pin instead of +//! refused. +//! +//! The `--hash` is written only when the requirements tree is already in +//! pip's hash-checking mode ([`requires_hashes`]): any `--hash` on any line +//! turns that mode on for the whole install, so a hashed vendor line in an +//! unhashed tree would make pip refuse every other requirement (#376). A +//! path line cannot carry a `#sha256=` fragment instead; the committed +//! wheel is the repository's own content. //! //! Logical-line model: physical lines join on a trailing `\`; comments start //! at a `#` preceded by whitespace (or column 0) outside that. The dominant @@ -19,7 +26,7 @@ use std::path::{Path, PathBuf}; use crate::crawlers::python_crawler::canonicalize_pypi_name; use crate::utils::fs::{atomic_write_bytes_preserving_mode, read_regular_to_string}; use crate::utils::requirements::{ - hash_options, logical_lines, split_comment, strip_comment, vendor_tag, + hash_options, logical_lines, requires_hashes, split_comment, strip_comment, vendor_tag, }; use super::common::{detect_eol, refuse_symlinked}; @@ -117,7 +124,9 @@ pub(super) enum RequirementsTarget { /// The wheel path + sha256 the wired vendor line still pins — the /// very pin `pip install --require-hashes` verifies. The in-sync /// rebuild guard falls back to it when the state.json ledger has no - /// entry left for the patch. + /// entry left for the patch. An unhashed vendor line (written into + /// an unhashed requirements set) pins its path alone: the sha256 is + /// empty and the guard checks only the path. pin: Option<(String, String)>, }, } @@ -192,16 +201,18 @@ fn vendored_uuid_for(content: &str, canon_name: &str) -> Option { /// Extract the (wheel path, sha256) pin the wired vendor line for /// `canon_name` carries — the same line shape [`vendored_uuid_for`] matches, -/// restricted to THIS patch uuid and requiring the `--hash=sha256:` pin -/// vendor always writes. Paths are returned bare (no `./` prefix), matching -/// the ledger's `artifact.path` spelling. +/// restricted to THIS patch uuid. A line with no `--hash` (vendor writes +/// none into an unhashed requirements set) still pins its path, with an +/// empty sha256; a `--hash` that is not a sha256 hex digest pins nothing. +/// Paths are returned bare (no `./` prefix), matching the ledger's +/// `artifact.path` spelling. fn wired_pin_in(content: &str, canon_name: &str, record_uuid: &str) -> Option<(String, String)> { vendor_lines(content, canon_name).find_map(|(parts, token, code)| { if parts.uuid != record_uuid { return None; } - let sha = hash_options(&code).into_iter().next()?; - if sha.len() != 64 || !sha.bytes().all(|b| b.is_ascii_hexdigit()) { + let sha = hash_options(&code).into_iter().next().unwrap_or_default(); + if !sha.is_empty() && (sha.len() != 64 || !sha.bytes().all(|b| b.is_ascii_hexdigit())) { return None; } let path = token.strip_prefix("./").unwrap_or(&token); @@ -440,6 +451,9 @@ async fn plan_requirements( wheel_sha256_hex: &str, ) -> Result, (&'static str, String)> { let files = collect_requirements_files(root).await?; + // pip's hash-checking mode spans the whole install: every reachable + // file (includes too) decides whether the vendor line is hashed. + let hashed = files.iter().any(|f| requires_hashes(&f.content)); let mut planned: Vec = Vec::new(); let mut rewrote_any = false; @@ -497,7 +511,7 @@ async fn plan_requirements( for (start, count, marker, _) in spans.iter().rev() { let line = vendor_line( rel_wheel, - wheel_sha256_hex, + hashed.then_some(wheel_sha256_hex), canon_name, version, marker, @@ -542,7 +556,7 @@ async fn plan_requirements( .expect("collect_requirements_files always yields the root file first"); let line = vendor_line( rel_wheel, - wheel_sha256_hex, + hashed.then_some(wheel_sha256_hex), canon_name, version, &None, @@ -572,15 +586,17 @@ async fn plan_requirements( Ok(planned) } -/// The committed vendor line. `transitive` adds the `(transitive)` note so a -/// reader knows the line was appended (no pin was replaced). +/// The committed vendor line. `sha256_hex` is the `--hash` pin, `None` for a +/// requirements tree outside hash-checking mode (module docs). `transitive` +/// adds the `(transitive)` note so a reader knows the line was appended (no +/// pin was replaced). /// /// Visible to the rest of `vendor` so the lockfile inventory's round-trip /// test can read back exactly what this writes (the two grammars — the one /// that writes a vendored line and the one that reads it — must agree). pub(in crate::vendor) fn vendor_line( rel_wheel: &str, - sha256_hex: &str, + sha256_hex: Option<&str>, canon_name: &str, version: &str, marker: &Option, @@ -590,9 +606,12 @@ pub(in crate::vendor) fn vendor_line( .as_ref() .map(|m| format!(" ; {m}")) .unwrap_or_default(); + let hash_part = sha256_hex + .map(|hex| format!(" --hash=sha256:{hex}")) + .unwrap_or_default(); let note = if transitive { " (transitive)" } else { "" }; format!( - "./{rel_wheel}{marker_part} --hash=sha256:{sha256_hex} # socket-patch vendor: {canon_name}=={version}{note}" + "./{rel_wheel}{marker_part}{hash_part} # socket-patch vendor: {canon_name}=={version}{note}" ) } @@ -911,6 +930,11 @@ mod tests { format!("./{REL_WHEEL} --hash=sha256:{SHA} # socket-patch vendor: six==1.16.0") } + /// [`expected_line`] for a requirements tree outside hash-checking mode. + fn expected_unhashed_line() -> String { + format!("./{REL_WHEEL} # socket-patch vendor: six==1.16.0") + } + async fn write_root(content: &str) -> tempfile::TempDir { let tmp = tempfile::tempdir().unwrap(); tokio::fs::write(tmp.path().join("requirements.txt"), content) @@ -1020,7 +1044,7 @@ mod tests { .unwrap(); assert_eq!( read_root(tmp.path()).await, - format!("requests==2.31.0\n{}\n", expected_line()) + format!("requests==2.31.0\n{}\n", expected_unhashed_line()) ); assert_eq!(wiring.len(), 1); assert_eq!(wiring[0].kind, "requirements_line"); @@ -1059,6 +1083,69 @@ mod tests { assert_eq!(read_root(tmp.path()).await, original); } + /// #376: an unhashed requirements set must stay unhashed. pip turns + /// hash-checking mode on for the whole install as soon as one line has a + /// `--hash`, so a hashed vendor line makes `pip install -r` refuse every + /// other (unhashed) requirement. A bare path cannot carry a `#sha256=` + /// fragment, so the committed wheel's line goes without one. + #[tokio::test] + async fn unhashed_requirements_get_an_unhashed_vendor_line() { + for (original, wired) in [ + ( + "six==1.16.0\nidna==3.7\n", + format!("./{REL_WHEEL} # socket-patch vendor: six==1.16.0\nidna==3.7\n"), + ), + ( + "idna==3.7\n", + format!( + "idna==3.7\n./{REL_WHEEL} # socket-patch vendor: six==1.16.0 (transitive)\n" + ), + ), + ] { + let tmp = write_root(original).await; + let wiring = wire_requirements(tmp.path(), "six", "1.16.0", REL_WHEEL, SHA) + .await + .unwrap(); + assert_eq!(read_root(tmp.path()).await, wired); + // Still read back as our line for this patch, pinning the path. + assert!(matches!( + preflight_requirements(tmp.path(), "six", "1.16.0", UUID).await, + Ok(RequirementsTarget::InSync { pin: Some((path, sha)) }) + if path == REL_WHEEL && sha.is_empty() + )); + let outcome = revert_requirements(&entry_for(wiring), tmp.path(), false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert_eq!(read_root(tmp.path()).await, original); + } + } + + /// #376: hashes anywhere in the requirements tree (an `-r` include, or + /// `--require-hashes`) mean pip is in hash-checking mode, so the vendor + /// line keeps its `--hash` pin. + #[tokio::test] + async fn hashes_in_an_include_keep_the_vendor_line_hashed() { + let tmp = write_root("-r deps.txt\nsix==1.16.0\n").await; + tokio::fs::write(tmp.path().join("deps.txt"), "idna==3.7 --hash=sha256:aa\n") + .await + .unwrap(); + wire_requirements(tmp.path(), "six", "1.16.0", REL_WHEEL, SHA) + .await + .unwrap(); + assert_eq!( + read_root(tmp.path()).await, + format!("-r deps.txt\n{}\n", expected_line()) + ); + + let tmp = write_root("--require-hashes\nsix==1.16.0\n").await; + wire_requirements(tmp.path(), "six", "1.16.0", REL_WHEEL, SHA) + .await + .unwrap(); + assert_eq!( + read_root(tmp.path()).await, + format!("--require-hashes\n{}\n", expected_line()) + ); + } + #[tokio::test] async fn marker_is_carried_over_verbatim() { let tmp = write_root("six==1.16.0 ; python_version >= \"3.8\"\n").await; @@ -1068,7 +1155,7 @@ mod tests { assert_eq!( read_root(tmp.path()).await, format!( - "./{REL_WHEEL} ; python_version >= \"3.8\" --hash=sha256:{SHA} # socket-patch vendor: six==1.16.0\n" + "./{REL_WHEEL} ; python_version >= \"3.8\" # socket-patch vendor: six==1.16.0\n" ) ); } @@ -1082,7 +1169,7 @@ mod tests { assert_eq!( read_root(tmp.path()).await, format!( - "python-dateutil==2.8.2\n./{REL_WHEEL} --hash=sha256:{SHA} # socket-patch vendor: six==1.16.0 (transitive)\n" + "python-dateutil==2.8.2\n./{REL_WHEEL} # socket-patch vendor: six==1.16.0 (transitive)\n" ) ); assert_eq!(wiring[0].action, WiringAction::Added); @@ -1115,7 +1202,7 @@ mod tests { tokio::fs::read_to_string(tmp.path().join("deps/pinned.txt")) .await .unwrap(), - format!("{}\n", expected_line()) + format!("{}\n", expected_unhashed_line()) ); assert_eq!(wiring.len(), 1); assert_eq!(wiring[0].file, "deps/pinned.txt"); @@ -1225,7 +1312,7 @@ mod tests { ); assert_eq!( read_root(tmp.path()).await, - format!("# vendored from C:\\deps\\\n{}\n", expected_line()) + format!("# vendored from C:\\deps\\\n{}\n", expected_unhashed_line()) ); } @@ -1373,8 +1460,12 @@ mod tests { let wiring = wire_requirements(tmp.path(), "six", "1.16.0", REL_WHEEL, SHA) .await .unwrap(); - // Drift: the user edited the vendor line (changed the hash). - let drifted = read_root(tmp.path()).await.replace(SHA, &"0".repeat(64)); + // Drift: the user edited the vendor line (added a marker). + let drifted = read_root(tmp.path()).await.replace( + " # socket-patch vendor", + " ; python_version >= \"3\" # socket-patch vendor", + ); + assert_ne!(drifted, read_root(tmp.path()).await); tokio::fs::write(tmp.path().join("requirements.txt"), &drifted) .await .unwrap(); @@ -1457,7 +1548,7 @@ mod tests { ); assert_eq!( read_root(tmp.path()).await, - format!("{}\n", expected_line()) + format!("{}\n", expected_unhashed_line()) ); // The BOM travels inside the replaced physical line's record, so @@ -1496,7 +1587,7 @@ mod tests { tokio::fs::read_to_string(tmp.path().join("dev.txt")) .await .unwrap(), - format!("{}\n", expected_line()) + format!("{}\n", expected_unhashed_line()) ); } @@ -1603,7 +1694,7 @@ mod tests { .unwrap(); assert_eq!(wiring.len(), 2); let written = read_root(tmp.path()).await; - assert_eq!(written.matches(&expected_line()).count(), 2); + assert_eq!(written.matches(&expected_unhashed_line()).count(), 2); let outcome = revert_requirements(&entry_for(wiring), tmp.path(), false).await; assert!(outcome.success, "{:?}", outcome.error); @@ -1675,6 +1766,12 @@ mod tests { assert_eq!(wired_pin_in(&content, "six", "not-the-uuid"), None); let short = content.replace(&hex, "abc"); assert_eq!(wired_pin_in(&short, "six", UUID), None); + // An unhashed vendor line still pins its path, with no sha256. + let unhashed = format!("{wheel} # socket-patch vendor: six==1.16.0\nattrs==23.1.0\n"); + assert_eq!( + wired_pin_in(&unhashed, "six", UUID), + Some((wheel.trim_start_matches("./").to_string(), String::new())) + ); } /// Multi-package coexistence: a root already carrying ANOTHER package's @@ -1856,7 +1953,7 @@ mod tests { assert_eq!( read_root(tmp.path()).await, format!( - "requests==2.31.0\n./{REL_WHEEL} --hash=sha256:{SHA} # socket-patch vendor: six==1.16.0 (transitive)\n" + "requests==2.31.0\n./{REL_WHEEL} # socket-patch vendor: six==1.16.0 (transitive)\n" ) ); let outcome = revert_requirements(&entry_for(wiring), tmp.path(), false).await; @@ -1878,7 +1975,7 @@ mod tests { assert_eq!( read_root(tmp.path()).await, format!( - "requests==2.31.0\r\nzope.interface==5.0\r\n./{REL_WHEEL} --hash=sha256:{SHA} # socket-patch vendor: six==1.16.0 (transitive)\r\n" + "requests==2.31.0\r\nzope.interface==5.0\r\n./{REL_WHEEL} # socket-patch vendor: six==1.16.0 (transitive)\r\n" ) ); } @@ -1918,7 +2015,7 @@ mod tests { tokio::fs::read_to_string(tmp.path().join("deps/b.txt")) .await .unwrap(), - format!("{}\n", expected_line()) + format!("{}\n", expected_unhashed_line()) ); // No transitive duplicate at the root, and the other files are // byte-untouched — the walk really visited them. @@ -2014,7 +2111,7 @@ mod tests { tokio::fs::read_to_string(tmp.path().join("dev.txt")) .await .unwrap(), - format!("{}\n", expected_line()) + format!("{}\n", expected_unhashed_line()) ); } diff --git a/crates/socket-patch-core/tests/fixtures/redirect/pypi/requirements/basic/expected-edits.json b/crates/socket-patch-core/tests/fixtures/redirect/pypi/requirements/basic/expected-edits.json index 1c218ec53..d83746fbd 100644 --- a/crates/socket-patch-core/tests/fixtures/redirect/pypi/requirements/basic/expected-edits.json +++ b/crates/socket-patch-core/tests/fixtures/redirect/pypi/requirements/basic/expected-edits.json @@ -5,6 +5,6 @@ "action": "rewritten", "key": "Requests", "original": "requests==2.28.1 ; python_version >= \"3.7\"", - "new": "Requests @ https://patch.socket.dev/patch/pypi/requests/2.28.1/11111111-1111-1111-1111-111111111111/33333333-3333-3333-3333-333333333333/requests-2.28.1-py3-none-any.whl ; python_version >= \"3.7\" --hash=sha256:deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef" + "new": "Requests @ https://patch.socket.dev/patch/pypi/requests/2.28.1/11111111-1111-1111-1111-111111111111/33333333-3333-3333-3333-333333333333/requests-2.28.1-py3-none-any.whl#sha256=deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef ; python_version >= \"3.7\"" } ] diff --git a/crates/socket-patch-core/tests/fixtures/redirect/pypi/requirements/basic/expected/requirements.txt b/crates/socket-patch-core/tests/fixtures/redirect/pypi/requirements/basic/expected/requirements.txt index c7f822106..0c789d68e 100644 --- a/crates/socket-patch-core/tests/fixtures/redirect/pypi/requirements/basic/expected/requirements.txt +++ b/crates/socket-patch-core/tests/fixtures/redirect/pypi/requirements/basic/expected/requirements.txt @@ -1,2 +1,2 @@ flask==2.0.1 -Requests @ https://patch.socket.dev/patch/pypi/requests/2.28.1/11111111-1111-1111-1111-111111111111/33333333-3333-3333-3333-333333333333/requests-2.28.1-py3-none-any.whl ; python_version >= "3.7" --hash=sha256:deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef +Requests @ https://patch.socket.dev/patch/pypi/requests/2.28.1/11111111-1111-1111-1111-111111111111/33333333-3333-3333-3333-333333333333/requests-2.28.1-py3-none-any.whl#sha256=deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef ; python_version >= "3.7" diff --git a/crates/socket-patch-core/tests/fixtures/vex-discover-golden/redirect-pypi.json b/crates/socket-patch-core/tests/fixtures/vex-discover-golden/redirect-pypi.json index b601a1156..5082f40ce 100644 --- a/crates/socket-patch-core/tests/fixtures/vex-discover-golden/redirect-pypi.json +++ b/crates/socket-patch-core/tests/fixtures/vex-discover-golden/redirect-pypi.json @@ -9,7 +9,7 @@ "artifact_rel": null, "locked_integrity": "Sha256Hex(\"deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef\")", "integrity_required": true, - "url": "https://patch.socket.dev/patch/pypi/requests/2.28.1/11111111-1111-1111-1111-111111111111/33333333-3333-3333-3333-333333333333/requests-2.28.1-py3-none-any.whl", + "url": "https://patch.socket.dev/patch/pypi/requests/2.28.1/11111111-1111-1111-1111-111111111111/33333333-3333-3333-3333-333333333333/requests-2.28.1-py3-none-any.whl#sha256=deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef", "lockfile_basis_ok": true } ],