Fix pylock projects dropping requirements.txt patch (#1758)

crates/socket-patch-cli/CLI_CONTRACT.md

@@ -1581,7 +1581,7 @@ command's integration tests check its output.
 | `redirect_unconfirmed`    | `skipped` (`details.mode: "hosted"`) | hosted `scan` / `get` (v5.0, additive): the patch was granted but no lockfile entry pinning it could be rewritten. The status and exit code are unchanged for now, pending the open hosted exit-policy decision (#704); `--silent` hides the human line. |
 | `lockfile_unreadable` / `lockfile_unparseable` / `patched_ref_invalid` / `patched_ref_unattributable` | run-level `warnings[]` | vex (every form): lockfile-discovery diagnostics — see "Manifest-less VEX (lockfile discovery)". Never flip the exit on their own. |
 | `vex_npm_shrinkwrap_only` | run warning and `failed[].reason` | vex (every form, #899): the patch is wired only in a root `npm-shrinkwrap.json` with no `package-lock.json` twin, which npm >= 12 never reads; no statement until the twin exists. `list` / `rollback` / `remove` still manage the wiring. |
-| `vendor_multiple_lockfiles` / `pypi_multiple_lockfiles` | `skipped` (warning) | vendor: a sibling lockfile of another package manager (for PyPI, also a root `requirements.txt` that pins the package beside the wired tool lock) will still install UNPATCHED bytes; names the wired winner + the ignored locks. Beside `Pipfile.lock` or `uv.lock`, an exact registry pin of the package in `requirements.txt` or an in-root `-r` include (`pipenv requirements` or `uv export` output) is wired with the lock instead and never named here (#612, #1368): both files then refer to the committed wheel, the ledger entry records both, every revert restores both, a superseding patch re-wires both, and a re-run over a project whose lock is already wired (a vendor from before, or an export made since) wires the export too (`pypi_requirements_sibling_wired`). A pin the requirements wiring cannot rewrite (a range, extras, an include outside the root) stays a loser. Likewise, a `pylock.toml` / `pylock.<name>.toml` that `pipenv lock` writes beside `Pipfile.lock` under `[pipenv] use_pylock = true` is wired with it (#1368), with the same ledger, revert, supersede and re-run behavior (`pypi_pipenv_pylock_wired`); a pylock the wiring cannot rewrite in place (a symlink, a non-registry source) stays a loser. |
+| `vendor_multiple_lockfiles` / `pypi_multiple_lockfiles` | `skipped` (warning) | vendor: a sibling lockfile of another package manager (for PyPI, also a root `requirements.txt` that pins the package beside the wired tool lock) will still install UNPATCHED bytes; names the wired winner + the ignored locks. Beside `Pipfile.lock`, `uv.lock` or a wired PEP 751 / script lock, an exact registry pin of the package in `requirements.txt` or an in-root `-r` include (`pipenv requirements` or `uv export` output, or the file `pip lock -r requirements.txt` read) is wired with the lock instead and never named here (#612, #1368, #1758): both files then refer to the committed wheel, the ledger entry records both, every revert restores both, a superseding patch re-wires both, and a re-run over a project whose lock is already wired (a vendor from before, or an export made since) wires the export too (`pypi_requirements_sibling_wired`). A pin the requirements wiring cannot rewrite (a range, extras, an include outside the root) stays a loser. Likewise, a `pylock.toml` / `pylock.<name>.toml` that `pipenv lock` writes beside `Pipfile.lock` under `[pipenv] use_pylock = true` is wired with it (#1368), with the same ledger, revert, supersede and re-run behavior (`pypi_pipenv_pylock_wired`); a pylock the wiring cannot rewrite in place (a symlink, a non-registry source) stays a loser. |
 | `vendor_npm_shrinkwrap_only` | `skipped` (warning) | vendor / scan / get `--mode vendored` (npm, #899): the package was wired into `npm-shrinkwrap.json`, the only npm lock (also on an in-sync re-run). npm >= 12 never reads the shrinkwrap and installs the unpatched registry bytes; rename the lock to `package-lock.json` (or commit a copy under that name) and re-run. |
 | `vendor_npm_deno_lock` | `skipped` (warning) | vendor / scan / get `--mode vendored` (npm, every lockfile flavor, #406): the package was vendored (also on an in-sync re-run), but the project root's `deno.lock` locks the same `name@version`. `deno install` installs a `package.json` project's npm dependencies from `deno.lock` and never reads the rewired lock, so installs Deno runs keep the registry copy and `vex` omits the patch (`vex_deno_lock_copy`); npm / pnpm / yarn / bun installs are patched. To patch what Deno installs, run `deno install` and then `socket-patch scan --mode agent`. |
 | `vendor_workspace_member_skipped` | `skipped` (warning) | vendor / scan / get `--mode vendored` (npm, #688): a lock entry with the package's `name@version` is the project's own source (a workspace member or a `file:` directory), so it is left alone while the lock's registry copies are vendored; patch that source directly if it needs the fix. When it is the only instance, vendoring refuses instead. |

crates/socket-patch-cli/tests/mode_migration_pypi.rs

@@ -3178,6 +3178,243 @@ async fn pipenv_hosted_to_vendored_keeps_the_requirements_patched() {
     );
 }
 
+/// `pip lock -r requirements.txt -o pylock.toml` (pip 26.2.2) over
+/// [`stage_pip_lock`]'s requirements.txt, byte for byte.
+const PIP_LOCK: &str = r#"lock-version = "1.0"
+created-by = "pip"
+
+[[packages]]
+name = "idna"
+version = "3.7"
+
+[[packages.wheels]]
+name = "idna-3.7-py3-none-any.whl"
+url = "https://files.pythonhosted.org/packages/e5/3e/741d8c82801c347547f8a2a06aa57dbb1992be9e948df2ea0eda2c8b79e8/idna-3.7-py3-none-any.whl"
+
+[packages.wheels.hashes]
+sha256 = "82fee1fc78add43492d3a1898bfa6d8a904cc97d8427f683ed8e798d07761aa0"
+
+[[packages]]
+name = "six"
+version = "1.16.0"
+
+[[packages.wheels]]
+name = "six-1.16.0-py2.py3-none-any.whl"
+url = "https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl"
+
+[packages.wheels.hashes]
+sha256 = "8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"
+"#;
+
+/// A `pip lock` project: the requirements.txt and the PEP 751 lock pip
+/// wrote from it, both pinning six (and idna); returns its wiring files.
+fn stage_pip_lock(root: &Path) -> &'static [&'static str] {
+    std::fs::write(root.join("requirements.txt"), "idna==3.7\nsix==1.16.0\n").unwrap();
+    std::fs::write(root.join("pylock.toml"), PIP_LOCK).unwrap();
+    &["requirements.txt", "pylock.toml"]
+}
+
+/// #1758: a pip project installs from both `requirements.txt` and the
+/// `pylock.toml` `pip lock` wrote from it. Vendored mode wires both to the
+/// same wheel (as hosted mode pins both), `vendor --check` is green, and
+/// the revert restores both files byte for byte. Before the fix only
+/// pylock.toml was wired and requirements.txt was named as an unpatched
+/// loser.
+#[tokio::test]
+async fn pip_lock_vendor_wires_requirements_with_the_pylock() {
+    let (_tmp, root) = project();
+    let files = stage_pip_lock(&root);
+    let originals: Vec<(&str, Vec<u8>)> = files
+        .iter()
+        .map(|f| (*f, std::fs::read(root.join(f)).unwrap()))
+        .collect();
+    stage_manifest(&root);
+    let needle = format!(".socket/vendor/pypi/{UUID}/");
+
+    // The dry run previews the requirements wiring and writes nothing.
+    let (code, env) = run_cli(&root, &["vendor", "--dry-run"], &[]);
+    assert_eq!(code, 0, "{env:#}");
+    let text = env.to_string();
+    assert!(
+        text.contains("pypi_requirements_sibling_wired")
+            && !text.contains("pypi_multiple_lockfiles"),
+        "{env:#}"
+    );
+    for (file, bytes) in &originals {
+        assert_eq!(&std::fs::read(root.join(file)).unwrap(), bytes, "{file}");
+    }
+
+    let (code, env) = run_cli(&root, &["vendor"], &[]);
+    assert_eq!(code, 0, "{env:#}");
+    for file in files {
+        assert!(
+            std::fs::read_to_string(root.join(file))
+                .unwrap()
+                .contains(&needle),
+            "{file} is wired: {env:#}"
+        );
+    }
+    assert!(
+        !env.to_string().contains("pypi_multiple_lockfiles"),
+        "requirements.txt is wired, not an unpatched loser: {env:#}"
+    );
+
+    let (code, env) = run_cli(&root, &["vendor", "--check"], &[]);
+    assert_eq!(code, 0, "both install sources are wired: {env:#}");
+
+    // Re-running is settled: nothing rewritten.
+    let wired: Vec<Vec<u8>> = files
+        .iter()
+        .map(|f| std::fs::read(root.join(f)).unwrap())
+        .collect();
+    let (code, env) = run_cli(&root, &["vendor"], &[]);
+    assert_eq!(code, 0, "{env:#}");
+    assert!(
+        !env.to_string().contains("pypi_multiple_lockfiles"),
+        "a settled re-run names no lockfile: {env:#}"
+    );
+    for (file, bytes) in files.iter().zip(&wired) {
+        assert_eq!(&std::fs::read(root.join(file)).unwrap(), bytes, "{file}");
+    }
+
+    let (code, env) = run_cli(&root, &["vendor", "--revert"], &[]);
+    assert_eq!(code, 0, "{env:#}");
+    for (file, bytes) in &originals {
+        assert_eq!(
+            &std::fs::read(root.join(file)).unwrap(),
+            bytes,
+            "{file} is restored"
+        );
+    }
+    assert!(!root.join(format!(".socket/vendor/pypi/{UUID}")).exists());
+}
+
+/// #1758: a pylock vendored while requirements.txt did not pin six yet
+/// (a project vendored before the fix wired only the pylock). Once the
+/// file pins six==1.16.0, the next `vendor` wires it to the committed
+/// wheel and records it in the ledger entry: `vendor --check` is green
+/// and `vendor --revert` restores both files.
+#[tokio::test]
+async fn pip_lock_rerun_wires_a_requirements_pin_added_later() {
+    let (_tmp, root) = project();
+    stage_pip_lock(&root);
+    std::fs::write(root.join("requirements.txt"), "idna==3.7\n").unwrap();
+    stage_manifest(&root);
+    let needle = format!(".socket/vendor/pypi/{UUID}/");
+    let (code, env) = run_cli(&root, &["vendor"], &[]);
+    assert_eq!(code, 0, "{env:#}");
+    assert_eq!(
+        std::fs::read_to_string(root.join("requirements.txt")).unwrap(),
+        "idna==3.7\n",
+        "{env:#}"
+    );
+
+    std::fs::write(root.join("requirements.txt"), "idna==3.7\nsix==1.16.0\n").unwrap();
+    let (code, env) = run_cli(&root, &["vendor"], &[]);
+    assert_eq!(code, 0, "{env:#}");
+    assert!(
+        env.to_string().contains("pypi_requirements_sibling_wired"),
+        "{env:#}"
+    );
+    assert!(
+        std::fs::read_to_string(root.join("requirements.txt"))
+            .unwrap()
+            .contains(&needle),
+        "{env:#}"
+    );
+    let (code, env) = run_cli(&root, &["vendor", "--check"], &[]);
+    assert_eq!(code, 0, "{env:#}");
+
+    let (code, env) = run_cli(&root, &["vendor", "--revert"], &[]);
+    assert_eq!(code, 0, "{env:#}");
+    assert_eq!(
+        std::fs::read(root.join("requirements.txt")).unwrap(),
+        b"idna==3.7\nsix==1.16.0\n",
+        "{env:#}"
+    );
+    assert_eq!(
+        std::fs::read(root.join("pylock.toml")).unwrap(),
+        PIP_LOCK.as_bytes(),
+        "{env:#}"
+    );
+}
+
+/// #1758: hosted mode pins six in both `requirements.txt` and the
+/// `pip lock` pylock.toml. The hosted → vendored switch (plain `vendor`,
+/// the eject, and `scan --mode vendored`) restores both files to their
+/// registry entry and must then wire BOTH to the vendored wheel. Before
+/// the fix only pylock.toml was wired, so `pip install -r
+/// requirements.txt` went from patched to unpatched while the run exited
+/// 0, and the run's own `vendor --check` failed on contested wiring.
+#[tokio::test]
+async fn pip_lock_hosted_to_vendored_keeps_the_requirements_patched() {
+    let needle = format!(".socket/vendor/pypi/{UUID}/");
+    for takeover in [false, true] {
+        // A fresh mock per lane: the fixture server's project routes are
+        // mounted per project root.
+        let server = MockServer::start().await;
+        let hosted_url = mount_hosted_api(&server, true).await;
+        mount_patch_view(&server).await;
+        let uri = server.uri();
+        let switch: Vec<&str> = if takeover {
+            vec![
+                "scan",
+                "--mode",
+                "vendored",
+                "--yes",
+                "--api-url",
+                &uri,
+                "--org",
+                ORG,
+                "--api-token",
+                "fake-token",
+                "--patch-server-url",
+                &uri,
+                "--vendor-url",
+                &uri,
+            ]
+        } else {
+            let mut args = vec!["vendor", "--patch-server-url", &uri, "--vendor-url", &uri];
+            args.extend(api_args(&uri));
+            args
+        };
+        let (_tmp, root) = project();
+        let files = stage_pip_lock(&root);
+        let (code, env) = hosted_scan(&root, &server);
+        assert_eq!(code, 0, "hosted scan: {env:#}");
+        for file in files {
+            let text = std::fs::read_to_string(root.join(file)).unwrap();
+            assert!(
+                text.contains(&hosted_url),
+                "hosted mode pins {file}:\n{text}\n{env:#}"
+            );
+        }
+
+        stage_manifest(&root);
+        prebuilt_common::mount_project(&server, &root).await;
+        let (code, env) = run_cli(&root, &switch, &[]);
+        assert_eq!(code, 0, "{switch:?}: {env:#}");
+        for file in files {
+            let text = std::fs::read_to_string(root.join(file)).unwrap();
+            assert!(
+                text.contains(&needle) && !text.contains(&hosted_url),
+                "{switch:?}: {file} is wired to the vendored wheel:\n{text}\n{env:#}"
+            );
+        }
+        assert!(
+            !env.to_string().contains("UNPATCHED")
+                && !env.to_string().contains("retain their existing sources"),
+            "{switch:?}: no install source is left unpatched: {env:#}"
+        );
+
+        let (code, env) = run_cli(&root, &["vendor", "--check"], &[]);
+        assert_eq!(
+            code, 0,
+            "{switch:?}: both install sources are wired: {env:#}"
+        );
+    }
+}
+
 /// The `pipenv requirements > requirements.txt` export of [`stage_pipenv`]'s
 /// lock: the file Docker / plain-pip installs read.
 const PIPENV_EXPORT: &str = "-i https://pypi.org/simple\nsix==1.16.0 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2'\n";