[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).
Summary
The package-name path of socket-patch get <name> matches the query against the crawled package names with a plain lowercase compare (fuzzy_match_packages). The Python crawler stores PyPI names in PEP 503 canonical form (typing-extensions, ruamel-yaml). So any spelling with _ or . never matches, even though pip, requirements.txt, PyPI's own project title and the .dist-info directory all use it: typing_extensions, ruamel.yaml, zope.interface, python_dateutil, Typing.Extensions.
get then prints No packages matching "ruamel.yaml" found. (JSON status: "no_match") and exits 0. Only the hyphenated canonical spelling works.
Impact
A user who copies the name from requirements.txt or pip list (ruamel.yaml, typing_extensions, zope.interface) is told the package isn't there and exits 0, so they conclude it can't be patched. Nothing gets patched, in every mode (hosted, vendored and agent). CI scripts that branch on status treat it as a clean "nothing to do".
This is a different code path from #910 (socket.yml specs and scan --package, policy::package_spec_matches). Draft PR #911 for #910 doesn't touch crawlers/fuzzy_match.rs.
Repro
This uses a mock patch API offering one free patch each for pkg:pypi/ruamel-yaml@0.18.6 and pkg:pypi/typing-extensions@4.12.2. Any API that has a patch for an installed package with _ / . in its name shows the same thing, because the miss happens before any API call.
python3 -m venv .venv
.venv/bin/pip install ruamel.yaml==0.18.6 typing_extensions==4.12.2
printf 'ruamel.yaml==0.18.6\ntyping_extensions==4.12.2\n' > requirements.txt
git init -q .
for spec in ruamel-yaml ruamel.yaml ruamel_yaml typing-extensions typing_extensions; do
VIRTUAL_ENV=$PWD/.venv socket-patch get "$spec" --mode hosted --yes --json \
--api-url "$MOCK" --org test-org --api-token x --patch-server-url "$MOCK" | jq -c '{status, found}'
done
Output on main 9c43dfc:
ruamel-yaml {"status":"success","found":1} # requirements.txt rewritten
ruamel.yaml {"status":"no_match","found":0} # exit 0, nothing written
ruamel_yaml {"status":"no_match","found":0}
typing-extensions {"status":"success","found":1}
typing_extensions {"status":"no_match","found":0}
The human output says Treating "ruamel.yaml" as a package name search / Found 5 packages / No packages matching "ruamel.yaml" found. --mode agent gives the same no_match for every non-canonical spelling, and so does Typing.Extensions. The mock log shows no search request at all for the failing spellings.
Expected vs actual
- Expected: PyPI names are case- and separator-insensitive (PEP 503; pip treats
ruamel.yaml, ruamel_yaml and ruamel-yaml as one project). socket-patch already canonicalises them everywhere it builds a purl (canonicalize_pypi_name), and get pkg:pypi/Typing_Extensions@4.12.2 works. CLI_CONTRACT.md describes this path as the installed-derived package-name search. A name pip would accept for an installed package should find it.
- Actual:
no_match, exit 0, for every spelling except the canonical hyphenated one.
Matrix
| OS |
pip / Python |
mode |
ruamel-yaml / typing-extensions |
ruamel.yaml / ruamel_yaml / typing_extensions / Typing.Extensions |
| Linux |
pip 24.0 / CPython 3.11 venv |
hosted |
success |
no_match (reproduced twice) |
| Linux |
pip 24.0 / CPython 3.11 venv |
agent |
found (1) |
no_match |
| macOS / Windows |
— |
— |
not probed |
OS-independent string match (same crawler output) |
First bad: the same lowercase-only compare is in v4.0.0 (crawlers/fuzzy_match.rs), so this isn't a recent regression.
Suspect code
crates/socket-patch-core/src/crawlers/fuzzy_match.rs:67 and :75-76: query.trim().to_lowercase() against pkg.name.to_lowercase(), with no PEP 503 canonicalisation of the query when the package is a PyPI one (canonicalize_pypi_name in crawlers/python_crawler.rs:46).
- Called from
crates/socket-patch-cli/src/commands/get.rs:2833.
Related: #910 / PR #911 (the same normalisation gap in policy::package_spec_matches), and #883 (one PyPI name module).
[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).
Summary
The package-name path of
socket-patch get <name>matches the query against the crawled package names with a plain lowercase compare (fuzzy_match_packages). The Python crawler stores PyPI names in PEP 503 canonical form (typing-extensions,ruamel-yaml). So any spelling with_or.never matches, even though pip,requirements.txt, PyPI's own project title and the.dist-infodirectory all use it:typing_extensions,ruamel.yaml,zope.interface,python_dateutil,Typing.Extensions.getthen printsNo packages matching "ruamel.yaml" found.(JSONstatus: "no_match") and exits 0. Only the hyphenated canonical spelling works.Impact
A user who copies the name from
requirements.txtorpip list(ruamel.yaml,typing_extensions,zope.interface) is told the package isn't there and exits 0, so they conclude it can't be patched. Nothing gets patched, in every mode (hosted, vendored and agent). CI scripts that branch onstatustreat it as a clean "nothing to do".This is a different code path from #910 (socket.yml specs and
scan --package,policy::package_spec_matches). Draft PR #911 for #910 doesn't touchcrawlers/fuzzy_match.rs.Repro
This uses a mock patch API offering one free patch each for
pkg:pypi/ruamel-yaml@0.18.6andpkg:pypi/typing-extensions@4.12.2. Any API that has a patch for an installed package with_/.in its name shows the same thing, because the miss happens before any API call.Output on main
9c43dfc:The human output says
Treating "ruamel.yaml" as a package name search/Found 5 packages/No packages matching "ruamel.yaml" found.--mode agentgives the sameno_matchfor every non-canonical spelling, and so doesTyping.Extensions. The mock log shows no search request at all for the failing spellings.Expected vs actual
ruamel.yaml,ruamel_yamlandruamel-yamlas one project). socket-patch already canonicalises them everywhere it builds a purl (canonicalize_pypi_name), andget pkg:pypi/Typing_Extensions@4.12.2works. CLI_CONTRACT.md describes this path as the installed-derived package-name search. A name pip would accept for an installed package should find it.no_match, exit 0, for every spelling except the canonical hyphenated one.Matrix
ruamel-yaml/typing-extensionsruamel.yaml/ruamel_yaml/typing_extensions/Typing.ExtensionsFirst bad: the same lowercase-only compare is in v4.0.0 (
crawlers/fuzzy_match.rs), so this isn't a recent regression.Suspect code
crates/socket-patch-core/src/crawlers/fuzzy_match.rs:67and:75-76:query.trim().to_lowercase()againstpkg.name.to_lowercase(), with no PEP 503 canonicalisation of the query when the package is a PyPI one (canonicalize_pypi_nameincrawlers/python_crawler.rs:46).crates/socket-patch-cli/src/commands/get.rs:2833.Related: #910 / PR #911 (the same normalisation gap in
policy::package_spec_matches), and #883 (one PyPI name module).