Fix Pipenv remedy for base-interpreter copies (#1450)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #1450

Summary

A --site-packages (PIPENV_SITE_PACKAGES=1) Pipenv venv with no copy of its own imports a package from its base interpreter (e.g. Debian's /usr/lib/python3/dist-packages). Since #1419 the hosted and vendored stale-install warnings fire for that copy, but they printed the usual Pipenv remedy: pipenv run pip uninstall -y <pkg> && pipenv sync, or pipenv --rm && pipenv sync. pip refuses to uninstall a copy outside the venv, and pipenv sync (also in a recreated --site-packages venv) counts the base copy as installed. Both left the package unpatched, and the next scan printed the same advice.

For a stale copy in a base-interpreter dir the venv inherits, the warning now prints an install over it inside the venv:

pipenv requirements[--dev | --categories "..."] > pipenv-locked.txt && pipenv run pip install --no-deps --ignore-installed -r pipenv-locked.txt

It also drops the "only protects fresh installs" claim, which is false on that host. A venv's own stale copy that hides the same release in its base interpreter gets the same install-over remedy, since uninstalling it would only expose the base copy to pipenv sync. Other stale copies in the venv keep the existing remedy, and when several kinds are stale each dir gets its own remedy.

Root cause

pypi_pipenv::stale_install_remedy built one remedy for every site dir. Both callers (scan/hosted/python.rs, vendor/pypi.rs) already held the stale dir but never asked whether it was the venv's own or an inherited base dir.

Changes

• crawlers/python_crawler.rs: new inherited_base_site_dirs(sites) returns the dirs among sites that a --system-site-packages venv in sites reaches in its base interpreter. It reuses the In a --system-site-packages venv, pip keeps the base interpreter's unpatched copy after a hosted rewrite, no stale-install warning fires, and vex attests it as patched #409 pyvenv.cfg/prefix logic.
• vendor/pypi_pipenv.rs: base_site_stale_install_remedy, with the requirements category args derived from the lock as sync's are. The lock-section computation is factored into lock_sections.
• Hosted and vendored warnings choose the remedy per stale dir. Hosted leaves out base dirs reached only by a Hatch env, and runs the probe only when a Pipenv package is involved.
• docs/testing/pipenv-compatibility.md: documents the base-copy case.

Test evidence

Issue	Regression test	Red without fix	Green with fix
#1450 (vendored)	vendor::pypi::tests::pipenv_stale_install_in_a_base_site_dir_installs_over_it	FAILED (panicked at pypi.rs:4275, generic remedy printed)	ok
#1450 (hosted)	commands::scan::hosted::python::tests::pipenv_base_site_copy_gets_an_install_over_it	FAILED (panicked at python.rs:453)	ok
helper	crawlers::python_crawler::tests::inherited_base_site_dirs_are_the_base_interpreters, vendor::pypi_pipenv::tests::base_site_stale_install_remedy_installs_over_the_base_copy	new API	ok

To get the red runs, I kept the new tests and stubbed the callers' inherited set to empty.

Real Pipenv 2026.8.0 check (Linux, py3.11, six 1.16.0 in /usr/lib/python3/dist-packages, PIPENV_SITE_PACKAGES=1):

• pipenv run pip uninstall -y six → Not uninstalling six at /usr/lib/python3/dist-packages, outside environment …. This reproduces the issue.
• The new command against a registry lock: the venv then imports <venv>/lib/python3.11/site-packages/six.py, and the base copy is unchanged.
• The new command against a vendored lock entry ("file": "./.socket/vendor/pypi/<uuid>/six-1.16.0-py2.py3-none-any.whl"): pipenv requirements exports the relative wheel path, pip installs it, and the venv imports its own copy.

Local runs:

• cargo fmt --all -- --check: ok.
• cargo clippy --workspace --all-features -- -D warnings: ok.
• cargo test -p socket-patch-cli --all-features --lib: 919 passed.
• cargo test -p socket-patch-core --all-features --lib: 6410 passed, 4 failed. The 4 failures are copy_tree::relax_loop_must_not_traverse_symlinked_root, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, pypi_poetry::wire_write_failure… and pypi_requirements::wire_failure_rolls_back…. Each relies on chmod 0555 making a write fail, which can't happen as uid 0 in this container. This diff doesn't touch them.
• --test in_process_redirect_pipenv --test mode_migration_pypi: 11 + 61 passed.
• A full cargo test --workspace ran out of the container's disk allowance, so CI covers the rest.

Review findings not fixed (/code-review high)

• Add --hash to pipenv requirements: I didn't. pipenv requirements --hash exports the vendored wheel line without its hash (Vendored Pipenv lock breaks pipenv requirements --hash | pip install -r on Pipenv 2023+: the vendored wheel is exported without a hash, so pip's hash-checking mode refuses the whole install #981; I confirmed this on 2026.8.0), so pip's hash-checking mode would refuse the whole vendored install. The hosted and vendored artifacts are the ones the lock already names.
• No remedy for Pipenv before 2022.4: I didn't add one. pipenv requirements doesn't exist there, and I couldn't verify that pipenv lock -r leaves the lock alone on those releases, so the text says "Pipenv 2022.4+" (2022.12+ when --categories is needed; I checked that 2022.4.8 lacks it and 2022.12.19 has it).
• Fold the two callers' remedy choice into one helper: I left this alone. The two warnings already word it differently ("rewritten" vs "wired" lock, vex --product <purl> vs vex), and unifying them would change unrelated text.
• Recompute per purl in vendored mode: I left this as is. stale_install_sites already recomputes the search order per purl the same way. It's a handful of pyvenv.cfg reads.

Bugbot round 1 (558051a)

• Shared Hatch/Pipenv `.venv` dropped from the hosted base-dir check: fixed in 662432f, with a regression case.
• Remedy didn't say it reinstalls the whole category: fixed in 662432f.
• Hatch env co-inheriting the same base dir keeps the warning firing: answered, not changed. The Hatch env really is still unpatched, and Pipenv's remedy can't reach it (Hosted Hatch rewrite leaves an existing Hatch environment unpatched with no stale-install warning, and vex still attests not_affected #335).

Final review (662432f)

• Mixed case, where a venv copy shadows the same base release: fixed in 76d2473, with hosted and vendored tests.
• User site ordering: checked on CPython 3.11 and it matches In a --system-site-packages venv, pip keeps the base interpreter's unpatched copy after a hosted rewrite, no stale-install warning fires, and vex attests it as patched #409 (venv site-packages comes before the user site), so nothing to change.
• --hash: unchanged, for the reason above.

Follow-ups

• The issue notes that pip, uv and Poetry remedies may have the same base-copy problem. That's untested and out of scope here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YCidp2ZuBQK2S7rDd2fvBM

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

Ready for review at 662432fee2715798ef9f6b7877064ca2f616f7aa.

• CI: all green on this head (35 pass + 19 skipped), no failures.
• Bugbot: reviewed 662432f, no new issues; no unresolved review threads.
• Mergeable, no CHANGELOG.md changes.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Final review brief

What it does: When a --site-packages Pipenv venv inherits a stale copy from the base interpreter, the stale-install warning used to print pipenv run pip uninstall … && pipenv sync. That can't touch the base copy, and pipenv sync treats it as already installed. The new inherited_base_site_dirs reuses the #409 machinery to find those base dirs. For copies there, the hosted (redirect_pypi_stale_install) and vendored (pypi_pipenv_stale_install) warnings now print pipenv requirements … && pipenv run pip install --no-deps --ignore-installed -r …. Fixes #1450.

Risk: low. Only warning text changes; nothing is written to disk and no patch logic moves. The extra filesystem work runs only when Pipenv venvs are present.

Look here:

• python_crawler.rs:625:`` inherited_base_site_dirs, which uses the same `system_site_dirs_of_venv` list the crawler pushed, so path comparison is consistent.
• hosted/python.rs:78 and :153:```` the hosted branch.
• pypi.rs:780: the vendored base/own split.
• pypi_pipenv.rs:842: the new remedy text.
• hosted/python.rs:480:````` the mixed-case assertion (see the first open question).

Verified:

• Read the full diff and the In a --system-site-packages venv, pip keeps the base interpreter's unpatched copy after a hosted rewrite, no stale-install warning fires, and vex attests it as patched #409 site-dir helpers.
• Layouts: Windows Lib\site-packages and Unix lib/pythonX.Y/site-packages both resolve, since venv_root_of_site looks 3 ancestors up.
• Hatch: the shared .venv and Hatch-only base dirs are handled and tested.
• The new tests fail without the change.
• CI on 662432f: 35 success, 19 skipped of 54; ci-ok and clippy green. Cursor Bugbot passed. No unresolved review threads. CHANGELOG.md untouched.

Changes I made: none.

Open questions (non-blocking; the PR fixes the reported base-only case and regresses nothing):

1. Mixed case keeps a remedy that can't work. The venv has its own stale six==1.16.0 and the base has 1.16.0. Only the venv copy is reported, with uninstall && pipenv sync. After the uninstall, sync sees the base 1.16.0 as installed, which is the PR's own premise, so the venv falls back to the unpatched base copy. The comment and test at hosted/python.rs:480 assert the opposite. A follow-up could use the --ignore-installed remedy whenever an own copy shadows a same-version inherited copy.
2. User site. ~/.local/.../site-packages is counted as base, but CPython puts it ahead of the venv on sys.path, so installing over it in the venv doesn't win.
3. Hashes. pipenv requirements without --hash drops the lock's hash checking, which pipenv sync had.

Auto-merge (squash) is armed, so approving sends it straight to the merge queue.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] Replies to the open questions in the final review:

1. Mixed case: fixed in 76d2473. A venv's own stale copy that hides the same release in its base interpreter now gets the install-over remedy, in both the hosted and vendored warnings (inherited_same_release and shadowing_stale_install_remedy). A different base release keeps the uninstall-and-sync remedy, since pipenv sync installs the locked release over that one. The test at hosted/python.rs that asserted the old remedy now covers both cases, and the vendored test has the same case.
2. User site: not a problem, as far as I can see. I checked on CPython 3.11: a --system-site-packages venv's sys.path lists <venv>/lib/python3.11/site-packages first, then ~/.local/lib/python3.11/site-packages, then /usr/local/lib/python3.11/dist-packages and /usr/lib/python3/dist-packages. That matches In a --system-site-packages venv, pip keeps the base interpreter's unpatched copy after a hosted rewrite, no stale-install warning fires, and vex attests it as patched #409's order, so a copy installed into the venv shadows the user-site copy, and the install-over remedy works for it.
3. Hashes: this one is intentional; the PR description has the reasoning. pipenv requirements --hash drops the vendored wheel line's hash (Vendored Pipenv lock breaks pipenv requirements --hash | pip install -r on Pipenv 2023+: the vendored wheel is exported without a hash, so pip's hash-checking mode refuses the whole install #981), so --require-hashes would refuse the vendored install. The artifacts are the ones the lock already names.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 76d2473. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Final review brief (updated for 76d2473)

What it does: A --site-packages Pipenv venv can inherit a stale copy from the base interpreter. The stale-install warning used to print pip uninstall … && pipenv sync for it, which can't fix a base copy. It now prints an install-over remedy instead (pipenv requirements > … && pipenv run pip install --no-deps --ignore-installed -r …), in both the hosted and the vendored warnings. The new commit 76d2473 also handles the mixed case my earlier brief raised: a venv's own stale copy that hides the same release in the base interpreter now gets the install-over remedy too, because uninstalling it would only expose the base copy. Fixes #1450.

Risk: low. Only warning text changes; nothing is written to disk. A case that doesn't match exactly falls back to the old remedy.

Look here:

• python_crawler.rs:645-668:`` inherited_same_release (new).
• hosted/python.rs:153-181:``` branch order, inherited then shadowing then plain.
• pypi.rs:779-826: how the vendored path splits shadowing, plain and base dirs, plus the lead text.
• pypi_pipenv.rs:853-890:```` shadowing_stale_install_remedy / `install_over_command`.
• Tests: hosted/python.rs:497-531 and pypi.rs:4378-4414.

Verified:

• Re-reviewed the new commit in the context of the full PR.
• Ran cargo test -p socket-patch-core --all-features --lib pipenv (108 pass) and cargo test -p socket-patch-cli --all-features --lib pipenv_ (4 pass).
• The new assertions fail without the commit. The different-release case keeps uninstall-and-sync.
• Names are canonicalized on both sides of the comparison, and a purl that doesn't parse never matches.
• No debug leftovers. CHANGELOG.md untouched.
• CI on 76d2473: ci-ok and clippy green. Cursor Bugbot passed. No unresolved threads.

Changes I made: none. I disarmed auto-merge when 76d2473 landed after the earlier brief, and I re-armed it with this brief.

Open questions (non-blocking):

1. The new same-release check at hosted/python.rs:164 isn't gated on is_global() / local-venv the way the base-dir list is. This only affects the remedy wording.
2. docs/testing/pipenv-compatibility.md:59-69 doesn't mention the shadowing case yet.
3. Earlier open questions 2 (user site) and 3 (pipenv requirements without --hash) still stand.

Auto-merge (squash) is armed, so approving sends it straight to the merge queue.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Still Ready for review at 76d2473957 (burn-down agent re-check after the mixed-case fix): CI 35/35 green + 19 path-skipped, Bugbot reviewed 76d2473 with no findings, no unresolved threads, mergeable, CHANGELOG.md untouched.

---

Generated by Claude Code