Vendored pip: a six==${SIXV} pin in requirements.txt (pip expands ${VAR} on every line) doesn't contest a pylock.toml wiring, so vendor --check passes and vex attests not_affected while pip install -r installs the unpatched release #1873

[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).

Summary

pip expands ${NAME} environment variables on every line of a requirements file (req_file.py expand_env_variables, run before a line is parsed), so six==${SIXV} with SIXV=1.16.0 is an exact six==1.16.0 pin. socket-patch expands ${NAME} only for -r / -c include targets (#994 / #995, nested_file_with in vendor/pypi_requirements.rs). The requirement line itself is read unexpanded by both the vendored planner and VEX discovery.

When the project also has a pip lock pylock.toml, scan --mode vendored wires six in pylock.toml only. It leaves six==${SIXV} on PyPI and exits 0. After that:

• vendor --check prints pkg:pypi/six@1.16.0: committed artifact and wiring verified and exits 0.
• vex, run against a venv installed with pip install -r requirements.txt (unpatched six), attests pkg:pypi/six@1.16.0 as not_affected.

The scan's only hint is the generic pypi_multiple_lockfiles skip ("wiring pylock.toml; installs driven by requirements.txt retain their existing sources").

This is the same outcome as #1851 (out-of-root include), but the cause is different: the pin is in the root file. PR #1853 doesn't cover it (re-tested on its head 70b2147, same result).

Impact

Version-templated requirements (torch==${TORCH_VERSION}, pkg==${PKG_VERSION} set by CI or a Dockerfile ARG/ENV) are a common pip idiom. For any pip install -r requirements.txt consumer of such a project, the package stays on the vulnerable release. Meanwhile vendor --check and the VEX document report it as patched. The false not_affected attestation is the serious part.

Repro

This uses the patch-API mock from ledger entry 20261029T021000Z: batch / by-package / view / package grant pkg:pypi/six@1.16.0 and pkg:pypi/idna@3.7, and each artifact is the real PyPI wheel with SAFE = True appended. Everything else comes from real PyPI. pip 26.2.2 / CPython 3.12.

export SIXV=1.16.0
printf 'idna==3.7\nsix==${SIXV}\n' > requirements.txt
python -m venv .venv && .venv/bin/pip install -r requirements.txt
.venv/bin/pip lock -r requirements.txt -o pylock.toml
VIRTUAL_ENV=$PWD/.venv socket-patch scan --mode vendored --yes   # exit 0
#   skipped pkg:pypi/six@1.16.0 pypi_multiple_lockfiles: "wiring pylock.toml; installs driven by requirements.txt retain their existing sources"
#   requirements.txt: idna vendored, `six==${SIXV}` untouched; pylock.toml: six -> .socket/vendor/pypi/<uuid>/six-1.16.0-…whl
socket-patch vendor --check          # exit 0: "six@1.16.0: committed artifact and wiring verified"
python -m venv fresh && fresh/bin/pip install --no-cache-dir -r requirements.txt
fresh/bin/python -c 'import six; print(hasattr(six, "SAFE"))'          # False (upstream wheel)
VIRTUAL_ENV=$PWD/fresh socket-patch vex --product pkg:pypi/app@1 -O vex.json
#   statements[0].status = not_affected, subcomponents = [idna@3.7, six@1.16.0]

Controls (same file, same environment):

• No pylock.toml: scan --mode vendored refuses six with pypi_requirement_not_pinned, exit 1, and vex doesn't attest six. That fails closed, though the message is wrong, since pip reads the line as pinned.
• Hosted, with or without pylock: vex doesn't attest six (correct). Without pylock, the scan reports redirect_unconfirmed.
• -r ${INC}/base.txt with six==1.16.0 inside, plus pylock, vendored: the include is expanded (Lock-only requirements.txt discovery skips -r includes that are quoted, backslash-escaped or use ${VAR} (-r "dev reqs.txt", --requirement="dev.txt", -r ${DIR}/dev.txt), so the scan exits 0 with "No patches" while pip installs the include's unpatched pins #994), so six is wired in base.txt too, and a fresh install is patched (pass).

Expected vs actual

• Expected: crates/socket-patch-cli/CLI_CONTRACT.md "Contested locks": "a requirements line pinned in any spelling that can only install the wired release … contests it like ==1.2.3 does", and then vendor --check fails and vex drops the reference with patched_ref_unattributable. With SIXV=1.16.0, six==${SIXV} can only install 1.16.0. pip refuses the line when the variable is unset or empty, because it then stays literal. socket-patch already reads the process environment the way pip does for include paths, so the requirement line should be read the same way. Either:
  • expand it and treat it like six==1.16.0. Then the planner wires it as a sibling, or refuses it the way Vendored pip: a pin in a -r include outside the project root is ignored when a pylock.toml exists, so vendor --check passes and vex attests not_affected while pip install -r requirements.txt installs the unpatched release #1851 does, and discovery contests a pylock-only wiring; or
  • at minimum, treat any unexpandable name==${…} line for the wired name as contesting, and refuse rather than demote requirements.txt to a pypi_multiple_lockfiles loser.
• Actual: exit 0, vendor --check exit 0, and vex attests not_affected while pip installs unpatched six.

Matrix (Linux; the code paths are OS-independent)

No first-bad bisect: requirement lines have never been env-expanded. Only include targets have been, since #995.

Suspect code

• crates/socket-patch-core/src/vex/discover/pypi_other.rs:271 / :292: extract_requirements hands the raw split_comment code to registry_pin (:353). parse_requirement("six==${SIXV}") yields no valid exact_version, so nothing contests the pylock wiring.
• crates/socket-patch-core/src/vendor/pypi_requirements.rs:129 (scan_pins) and :427 (sibling_pin_wirable): the planner reads the same raw line, finds no pin, and detect_pypi_flavor (vendor/pypi.rs:395 / :427) demotes requirements.txt to a pypi_multiple_lockfiles loser.
• crates/socket-patch-core/src/utils/requirements.rs:291 (expand_env_vars) already implements pip's expansion. It's only called from nested_file_with (vendor/pypi_requirements.rs:1548).
• Related: Vendored pip: a pin in a -r include outside the project root is ignored when a pylock.toml exists, so vendor --check passes and vex attests not_affected while pip install -r requirements.txt installs the unpatched release #1851 / PR Fix pylock wiring ignoring an out-of-root -r pin (#1851) #1853 (out-of-root include, same outcome), Hosted → vendored over an extras pin (six[foo]==X) in requirements.txt beside a pip lock pylock.toml still un-hosts requirements.txt and leaves it unpatched, exit 0 (#1758 residual) #1794 / PR Fix extras pin un-hosted beside a pylock (#1794) #1796 (extras pin beside pylock), Lock-only requirements.txt discovery skips -r includes that are quoted, backslash-escaped or use ${VAR} (-r "dev reqs.txt", --requirement="dev.txt", -r ${DIR}/dev.txt), so the scan exits 0 with "No patches" while pip installs the include's unpatched pins #994 (${VAR} includes).

No probe runs: this routine's probe branches are still blocked, so this was reproduced on Linux only.