Fix pip constraints files broken by link pins (#740)

crates/socket-patch-cli/CLI_CONTRACT.md

@@ -1614,6 +1614,7 @@ command's integration tests check its output.
 | `vendor_artifact_missing` | reason | The recorded artifact is missing; repair requires an online exact redownload. |
 | `vendor_artifact_corrupt` | reason | The artifact does not match the recorded hash or inventory; repair requires an online exact redownload. |
 | `vendor_artifact_reused` | `skipped` (verbose note) | vendor / scan `--mode vendored` (pypi): the wiring was dropped by a relock but the committed wheel the ledger vouches for verified, so it was re-wired as-is — no service download, no rebuild; the lock pins the first run's sha again. |
+| `vendor_requirements_constraints_link` | `skipped` (advisory) | vendor / scan `--mode vendored` (pypi / requirements.txt), dry run included: a requirements file the run wired (or would wire) a vendored wheel path into is read as `-c` / `--constraint` by a sibling requirements file (a `*.txt` / `*.in` in the project root or a root `requirements*/` directory, or a requirements-named `*.txt` / any `*.in` in another top-level directory, such as `docs/requirements.txt`; hidden and dependency directories are skipped). pip 20.3.x and 21.0.x refuse a link in a constraints file ("Links are not allowed as constraints"), so installing with that file fails on those pips. One advisory per reading file; the package is still vendored (#740). |
 | `vendor_redownload_failed` | `failed` | vendor: the same-UUID artifact could not be downloaded and verified against its original ledger. Existing files and fingerprints are preserved. |
 | `vendor_artifact_redownload_failed` | `failed` | repair: download unavailable, integrity mismatch, or downloaded bytes/inventory differ from the ledger. Existing files are preserved. |
 | `vendor_artifact_unrepairable` | `failed` | repair: the ledger identity or patch record cannot be trusted or recovered. |
@@ -1665,6 +1666,7 @@ command's integration tests check its output.
 | `redirect_takeover_not_pinned` | `skipped` (hosted event) | scan/get `--mode hosted`: the skip reason of a retracted takeover when no rewriter warning names the cause. |
 | `redirect_hatch_lock_regenerated` | `warnings[]` (warning) | scan/get `--mode hosted` (pypi / Hatch 1.17+): the project's Hatch environments are locked (`lock-envs = true`, an environment's `locked = true` or `lock-filename`), so its `pylock*.toml` is derived from pyproject and Hatch regenerates it on the next sync. The package is wired in the Hatch declarations as well as in the lock (rewriting only the lock was discarded by Hatch, #479); run `hatch dep lock` to refresh the lock (`hatch dep lock --check` reports it stale until then). Vendored routes such a project to the Hatch lane and its pip-installer refusal. |
 | `redirect_requirements_direct_reference` | `warnings[]` (warning) | scan/get `--mode hosted` (pypi / requirements.txt): the root `requirements.txt` installs the patched release from a PEP 508 direct reference the user wrote (`name @ <url>` whose archive names that release: a `files.pythonhosted.org` file, a private mirror, an internal fork build, a `file://` path), not from socket-patch's own hosted artifact. It is the user's own source choice, so the line is left byte-for-byte and the package is not redirected (#542). Before v5.0 the line was swapped for the hosted build, and rollback then wrote a plain `name==version` index pin. |
+| `redirect_requirements_constraints_link` | `warnings[]` (warning) | scan/get `--mode hosted` (pypi / requirements.txt): the root `requirements.txt` was rewritten, and a sibling requirements file (a `*.txt` / `*.in` in the project root or a root `requirements*/` directory, or a requirements-named `*.txt` / any `*.in` in another top-level directory, such as `docs/requirements.txt`; hidden and dependency directories are skipped) reads it as a constraints file (`-c` / `--constraint`). The rewritten pin is a URL, and pip 20.3.x and 21.0.x refuse a link in a constraints file ("Links are not allowed as constraints"), so installing with that file fails on those pips; pip 20.2 and 21.1+ accept it. One warning per reading file; the redirect still happens (#740). A constraints file named only on the pip command line is not detected. |
 | `redirect_requirements_takeover_unreachable` | `warnings[]` (warning) | scan/get `--mode hosted` (pypi / requirements.txt): a vendored → hosted takeover of a package that vendored mode wired through a pin in a `-r` include, or through a `(transitive)` line it appended to the root `requirements.txt`. Hosted mode only rewrites an existing pin in the root `requirements.txt`, so the staged takeover is retracted (wet and `--dry-run`): the vendored wiring, ledger entry and wheel stay byte-identical, the purl is skipped with this code as `skipped` (hosted event), and nothing is redirected for it. Exit 0. The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), move the pin from the include into the root `requirements.txt` and delete it from the include (or, for a `(transitive)` line, add an exact `==` pin to the root file), then re-run `scan --mode hosted`. |
 | `redirect_uv_takeover_version_unreachable` | `warnings[]` (warning) | scan/get `--mode hosted` (pypi / uv): a vendored → hosted takeover of a package whose recorded pre-vendor `uv.lock` entry is at another version than the patch (vendored uv pins the lock entry down to the patch's version, for example when the lock resolved a newer release). Reverting would bring the lock's own version back, and hosted mode only pins the version the lock resolves, so the staged takeover is retracted (wet and `--dry-run`): the vendored wiring, ledger entry and wheel stay byte-identical, the purl is skipped with this code as `skipped` (hosted event), and nothing is redirected for it. Exit 0. The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), make the project resolve the patch's version (for example an exact `==` requirement) and re-lock, then re-run `scan --mode hosted`. |
 | `redirect_vlt_missing_sha512` / `redirect_vlt_entry_not_found` / `redirect_vlt_entry_vendored` / `redirect_vlt_unsupported_lock_key` | `warnings[]` (warning) | scan/get `--mode hosted` (vlt): the grant has no sha512 / the lock has no default-registry node for `name@version` (not emitted when `redirect_vlt_custom_registry_skipped` already covers the dep's only nodes) / the only match is a vendored `file` node under `.socket/vendor/npm/<uuid>/` / a default-registry instance is outside vlt's node-line grammar or still unpatched after the splice. Per dep; none of the dep's instances is written. `redirect_vlt_missing_sha512` and `redirect_vlt_unsupported_lock_key` refuse the dep: it is never confirmed, whichever lock drives (a sibling lock may still carry its rewritten URL). `redirect_vlt_entry_not_found` and `redirect_vlt_entry_vendored` only say `vlt-lock.json` does not wire it: while vlt drives it is not confirmed; otherwise a sibling lock's rules may confirm it. Exit 0. |

crates/socket-patch-cli/tests/mode_migration_pypi.rs

@@ -433,6 +433,162 @@ async fn requirements_sole_pin_vendored_to_hosted() {
     assert_vendored_to_hosted(&root, &["requirements.txt"]).await;
 }
 
+/// #740: a sibling requirements file that reads the root requirements.txt
+/// as `-c` constraints breaks on pip 20.3.x / 21.0.x once the root pin is a
+/// link ("Links are not allowed as constraints"). Both writers still
+/// rewrite the pin, but now name every such reader in a warning: hosted
+/// `redirect_requirements_constraints_link`, vendored (dry run and wet)
+/// `vendor_requirements_constraints_link`. A line that names the file as
+/// `-r` (or `-r` after `-c`, which pip follows instead) is not a reader,
+/// and a project with no reader gets no warning.
+#[tokio::test]
+async fn requirements_read_as_constraints_warn_in_both_modes() {
+    fn stage(root: &Path, readers: bool) {
+        std::fs::write(root.join("requirements.txt"), "six==1.16.0\n").unwrap();
+        std::fs::write(root.join("include.txt"), "-r requirements.txt\n").unwrap();
+        std::fs::write(
+            root.join("both.txt"),
+            "-c requirements.txt -r requirements.txt\n",
+        )
+        .unwrap();
+        if readers {
+            std::fs::write(
+                root.join("requirements-dev.txt"),
+                "-c requirements.txt\nidna==3.7\n",
+            )
+            .unwrap();
+            std::fs::create_dir_all(root.join("requirements")).unwrap();
+            std::fs::write(
+                root.join("requirements/test.txt"),
+                "--constraint=../requirements.txt  # pins\n",
+            )
+            .unwrap();
+        }
+    }
+    fn reported(env: &Value, code: &str) -> Vec<String> {
+        let mut readers: Vec<String> = Vec::new();
+        let mut walk = vec![env];
+        while let Some(v) = walk.pop() {
+            match v {
+                Value::Object(map) => {
+                    if ["code", "errorCode"]
+                        .iter()
+                        .any(|key| map.get(*key).and_then(Value::as_str) == Some(code))
+                    {
+                        let text = v.to_string();
+                        for reader in [
+                            "requirements-dev.txt",
+                            "requirements/test.txt",
+                            "both.txt",
+                            "include.txt",
+                        ] {
+                            if text.contains(&format!("{reader} reads requirements.txt")) {
+                                readers.push(reader.to_string());
+                            }
+                        }
+                        assert!(
+                            text.contains("Links are not allowed as constraints"),
+                            "{text}"
+                        );
+                    }
+                    walk.extend(map.values());
+                }
+                Value::Array(items) => walk.extend(items),
+                _ => {}
+            }
+        }
+        readers.sort();
+        readers
+    }
+    let expected = vec![
+        "requirements-dev.txt".to_string(),
+        "requirements/test.txt".to_string(),
+    ];
+    for readers in [true, false] {
+        let want = if readers {
+            expected.clone()
+        } else {
+            Vec::new()
+        };
+
+        let (_tmp, root) = project();
+        stage(&root, readers);
+        let server = MockServer::start().await;
+        mount_hosted_api(&server, true).await;
+        let (code, env) = hosted_scan(&root, &server);
+        assert_eq!(code, 0, "{env:#}");
+        assert_eq!(
+            hosted_pinned(&env),
+            1,
+            "the pin is still redirected: {env:#}"
+        );
+        assert_eq!(
+            reported(&env, "redirect_requirements_constraints_link"),
+            want,
+            "{env:#}"
+        );
+        // A re-run rewrites nothing; the URL pin still breaks the readers.
+        let (code, env) = hosted_scan(&root, &server);
+        assert_eq!(code, 0, "{env:#}");
+        assert_eq!(
+            reported(&env, "redirect_requirements_constraints_link"),
+            want,
+            "re-run: {env:#}"
+        );
+
+        let (_tmp, root) = project();
+        stage(&root, readers);
+        stage_manifest(&root);
+        let (code, env) = run_cli(&root, &["vendor", "--dry-run"], &[]);
+        assert_eq!(code, 0, "{env:#}");
+        assert_eq!(
+            reported(&env, "vendor_requirements_constraints_link"),
+            want,
+            "dry run: {env:#}"
+        );
+        assert!(
+            !readers || env.to_string().contains("would pin six==1.16.0"),
+            "{env:#}"
+        );
+        let (code, env) = run_cli(&root, &["vendor"], &[]);
+        assert_eq!(code, 0, "{env:#}");
+        assert!(std::fs::read_to_string(root.join("requirements.txt"))
+            .unwrap()
+            .contains(&format!(".socket/vendor/pypi/{UUID}/")));
+        assert_eq!(
+            reported(&env, "vendor_requirements_constraints_link"),
+            want,
+            "{env:#}"
+        );
+        assert!(
+            !readers || env.to_string().contains("now pins six==1.16.0"),
+            "{env:#}"
+        );
+        let (code, env) = run_cli(&root, &["vendor"], &[]);
+        assert_eq!(code, 0, "{env:#}");
+        assert_eq!(
+            reported(&env, "vendor_requirements_constraints_link"),
+            want,
+            "in-sync re-run: {env:#}"
+        );
+        // An in-sync run that rebuilds a missing wheel keeps the wiring too.
+        std::fs::remove_dir_all(root.join(format!(".socket/vendor/pypi/{UUID}"))).unwrap();
+        let (code, env) = run_cli(&root, &["vendor"], &[]);
+        assert_eq!(code, 0, "{env:#}");
+        assert!(
+            env["events"]
+                .as_array()
+                .is_some_and(|events| events.iter().any(|e| e["action"] == "rebuilt")),
+            "{env:#}"
+        );
+        assert_eq!(
+            reported(&env, "vendor_requirements_constraints_link"),
+            want,
+            "in-sync rebuild: {env:#}"
+        );
+    }
+}
+
 /// #410: a requirements.txt in which every requirement is the hosted pin
 /// (a lone `six==1.16.0`, or one beside `-e .`) can be unwound again.
 /// Hosted `rollback`, `remove` and the hosted → vendored takeover restore

crates/socket-patch-core/src/hosted/engine.rs

@@ -28,6 +28,7 @@ use crate::constants::npm_family::{
     RUSH_COMMON_LOCK_REL, RUSH_SUBSPACES_DIR, VLT_HIDDEN_LOCK_REL, VLT_LOCK,
 };
 use crate::formats::cargo::source_replacement::CratesIoSource;
+use crate::formats::governing_locks::PYPI_REQUIREMENTS;
 use crate::patch::redirect::npmrc::{
     effective_replace_registry_host, plan_npmrc_allow_remote_with, replace_registry_host_rewrites,
     NpmrcPlan, OuterAllowRemote, NPMRC_ALLOW_REMOTE_EDIT_KIND, NPMRC_REL,
@@ -1765,6 +1766,8 @@ pub async fn rewrite(
         )
         .await;
         if vetoed.is_empty() {
+            let constraints = constraints_link_warnings(view, &done, &kept).await;
+            done.rewrite.warnings.extend(constraints);
             done.unattributed = unattributed;
             done.rewrite.warnings.extend(lockless);
             done.rewrite.warnings.extend(copy_left);
@@ -1777,6 +1780,48 @@ pub async fn rewrite(
     }
 }
 
+/// A `redirect_requirements_constraints_link` warning for each sibling
+/// requirements file that reads the root requirements.txt as `-c`
+/// constraints while a hosted pin there is a URL (#740), which pip 20.3
+/// and 21.0 refuse in a constraints file. Said on every run that confirms
+/// such a pin, not only the one that writes it: the hazard lasts as long
+/// as the pin does.
+async fn constraints_link_warnings(
+    view: &ProjectView<'_>,
+    done: &Rewritten,
+    kept: &[Candidate],
+) -> Vec<RewriteWarning> {
+    let confirmed = &done.rewrite.confirmed_requirements_uuids;
+    let mut pinned: Vec<&str> = kept
+        .iter()
+        .filter(|c| confirmed.contains(&c.dep.patch_uuid))
+        .map(|c| c.dep.name.as_str())
+        .collect();
+    pinned.sort_unstable();
+    pinned.dedup();
+    if pinned.is_empty() {
+        return Vec::new();
+    }
+    crate::vendor::pypi_requirements::constraint_readers_in(*view, &[PYPI_REQUIREMENTS.to_string()])
+        .await
+        .into_iter()
+        .map(|found| {
+            warning(
+                "redirect_requirements_constraints_link",
+                format!(
+                "{} reads {} as a constraints file (`-c`), and the hosted pin of {} in {} is a \
+                 URL; {}",
+                found.reader,
+                found.target,
+                pinned.join(", "),
+                found.target,
+                crate::vendor::pypi_requirements::CONSTRAINT_LINK_REMEDY
+            ),
+            )
+        })
+        .collect()
+}
+
 /// Lockfile discovery over the project as `done` would leave it, read as
 /// the management commands read it ([`HostedInventory`]): the skips for
 /// the confirmed candidates whose pin would be contested wiring, a