Skip to content

get <name> doesn't PEP 503-normalise PyPI names, so get typing_extensions or get ruamel.yaml reports "No packages matching" (exit 0) for an installed, patchable package #926

Description

[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).

Summary

The package-name path of socket-patch get <name> matches the query against the crawled package names with a plain lowercase compare (fuzzy_match_packages). The Python crawler stores PyPI names in PEP 503 canonical form (typing-extensions, ruamel-yaml). So any spelling with _ or . never matches, even though pip, requirements.txt, PyPI's own project title and the .dist-info directory all use it: typing_extensions, ruamel.yaml, zope.interface, python_dateutil, Typing.Extensions.

get then prints No packages matching "ruamel.yaml" found. (JSON status: "no_match") and exits 0. Only the hyphenated canonical spelling works.

Impact

A user who copies the name from requirements.txt or pip list (ruamel.yaml, typing_extensions, zope.interface) is told the package isn't there and exits 0, so they conclude it can't be patched. Nothing gets patched, in every mode (hosted, vendored and agent). CI scripts that branch on status treat it as a clean "nothing to do".

This is a different code path from #910 (socket.yml specs and scan --package, policy::package_spec_matches). Draft PR #911 for #910 doesn't touch crawlers/fuzzy_match.rs.

Repro

This uses a mock patch API offering one free patch each for pkg:pypi/ruamel-yaml@0.18.6 and pkg:pypi/typing-extensions@4.12.2. Any API that has a patch for an installed package with _ / . in its name shows the same thing, because the miss happens before any API call.

python3 -m venv .venv
.venv/bin/pip install ruamel.yaml==0.18.6 typing_extensions==4.12.2
printf 'ruamel.yaml==0.18.6\ntyping_extensions==4.12.2\n' > requirements.txt
git init -q .
for spec in ruamel-yaml ruamel.yaml ruamel_yaml typing-extensions typing_extensions; do
  VIRTUAL_ENV=$PWD/.venv socket-patch get "$spec" --mode hosted --yes --json \
    --api-url "$MOCK" --org test-org --api-token x --patch-server-url "$MOCK" | jq -c '{status, found}'
done

Output on main 9c43dfc:

ruamel-yaml        {"status":"success","found":1}     # requirements.txt rewritten
ruamel.yaml        {"status":"no_match","found":0}    # exit 0, nothing written
ruamel_yaml        {"status":"no_match","found":0}
typing-extensions  {"status":"success","found":1}
typing_extensions  {"status":"no_match","found":0}

The human output says Treating "ruamel.yaml" as a package name search / Found 5 packages / No packages matching "ruamel.yaml" found. --mode agent gives the same no_match for every non-canonical spelling, and so does Typing.Extensions. The mock log shows no search request at all for the failing spellings.

Expected vs actual

  • Expected: PyPI names are case- and separator-insensitive (PEP 503; pip treats ruamel.yaml, ruamel_yaml and ruamel-yaml as one project). socket-patch already canonicalises them everywhere it builds a purl (canonicalize_pypi_name), and get pkg:pypi/Typing_Extensions@4.12.2 works. CLI_CONTRACT.md describes this path as the installed-derived package-name search. A name pip would accept for an installed package should find it.
  • Actual: no_match, exit 0, for every spelling except the canonical hyphenated one.

Matrix

OS pip / Python mode ruamel-yaml / typing-extensions ruamel.yaml / ruamel_yaml / typing_extensions / Typing.Extensions
Linux pip 24.0 / CPython 3.11 venv hosted success no_match (reproduced twice)
Linux pip 24.0 / CPython 3.11 venv agent found (1) no_match
macOS / Windows — — not probed OS-independent string match (same crawler output)

First bad: the same lowercase-only compare is in v4.0.0 (crawlers/fuzzy_match.rs), so this isn't a recent regression.

Suspect code

  • crates/socket-patch-core/src/crawlers/fuzzy_match.rs:67 and :75-76: query.trim().to_lowercase() against pkg.name.to_lowercase(), with no PEP 503 canonicalisation of the query when the package is a PyPI one (canonicalize_pypi_name in crawlers/python_crawler.rs:46).
  • Called from crates/socket-patch-cli/src/commands/get.rs:2833.

Related: #910 / PR #911 (the same normalisation gap in policy::package_spec_matches), and #883 (one PyPI name module).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions