Skip to content

Wrong-shape JSON from a candidate aborts discovery #197

Closed

Description

@gaborbernat

A program named like Python that prints valid JSON of the wrong shape makes discovery raise instead of trying the next interpreter on PATH.

Affected code

src/python_discovery/_cached_py_info.py:267 at the reported commit. The reproducer also returns BUG on current main 004294957ed227fe7e52b2864952ba09fd28b220.

Mechanism

The probe parses JSON but uses its fields before checking that the result has the collector's expected shape.

Reproduction

Check out the reported commit or current main. Save the files below as named, then build and run:

docker build -t python-discovery-audit -f - . <<'DOCKERFILE'
FROM python:3.15-bookworm
RUN python -m venv /opt/venv
ENV PATH=/opt/venv/bin:$PATH
COPY . /src
WORKDIR /src
ENV SETUPTOOLS_SCM_PRETEND_VERSION=1.9.1
RUN pip install -e . --group test
DOCKERFILE
docker run --rm --network=none --memory=2g -u 0 -w /src \
  -v "$PWD/repro:/tmp/repro:ro" -v "$PWD/repro.d:/tmp/repro.d:ro" \
  python-discovery-audit /tmp/repro

repro

#!/usr/bin/env python3
import shutil
import subprocess
import tempfile
from pathlib import Path


def find_python311_with_first_on_path(candidate):
    """Run the embedder with `candidate` as the first python3.11 on PATH; the real /usr/bin/python3.11 comes after it."""
    with tempfile.TemporaryDirectory() as scratch:
        fake_python = Path(scratch, "bin", "python3.11")
        fake_python.parent.mkdir()
        shutil.copy(f"/tmp/repro.d/{candidate}", fake_python)
        fake_python.chmod(0o755)  # repro.d is mounted without exec bits

        embedder = subprocess.run(
            [
                "/opt/venv/bin/python",
                "/tmp/repro.d/find_python311.py",
                f"{fake_python.parent}:/usr/bin",
                f"{scratch}/cache",
            ],
            capture_output=True,
            text=True,
            timeout=60,
        )

        print(f"--- first python3.11 on PATH is {candidate}")
        print("it was probed as:")
        print(Path(f"{fake_python}.argv").read_text(), end="")
        print("get_interpreter() returned:", embedder.stdout.strip() or "nothing")
        print("embedder stderr, last lines:")
        print("\n".join(embedder.stderr.strip().splitlines()[-4:]))
        print()
        return embedder


# control: output that is not JSON is a failed probe; the candidate is skipped and /usr/bin/python3.11 is found
not_json = find_python311_with_first_on_path("candidate_not_json.sh")

# trigger: output that is valid JSON, but not the object _py_info_collect.py prints
empty_object = find_python311_with_first_on_path("candidate_empty_object.sh")
version_info_only = find_python311_with_first_on_path("candidate_version_info_only.sh")

# check: a JSON candidate must be skipped like the control, not end discovery with an exception
control_skipped = "exe=/usr/bin/python3.11," in not_json.stdout
empty_object_escaped = empty_object.stderr.rstrip().endswith("KeyError: 'version_info'")
version_info_only_escaped = version_info_only.stderr.rstrip().endswith(
    "AttributeError: 'PathPythonInfo' object has no attribute 'original_executable'"
)
print("BUG" if control_skipped and (empty_object_escaped or version_info_only_escaped) else "OK")

repro.d/candidate_empty_object.sh

#!/bin/sh
# a program named python3.11 that is not Python: ignores its arguments, prints a JSON object, exits 0
echo "$0 $*" >> "$0.argv"
echo '{}'

repro.d/candidate_not_json.sh

#!/bin/sh
# a program named python3.11 that is not Python: ignores its arguments, prints something that is not JSON, exits 0
echo "$0 $*" >> "$0.argv"
echo 'not json'

repro.d/candidate_version_info_only.sh

#!/bin/sh
# a program named python3.11 that is not Python: prints a JSON object with only version_info, the one key
# PythonInfo.from_dict() reads, exits 0
echo "$0 $*" >> "$0.argv"
echo '{"version_info": {"major": 3, "minor": 11, "micro": 0, "releaselevel": "final", "serial": 0}}'

repro.d/find_python311.py

"""An embedder such as virtualenv or tox: ask python-discovery for python3.11 on a given PATH and print what it chose."""

import sys
from pathlib import Path

from python_discovery import DiskCache, get_interpreter

search_path, cache_dir = sys.argv[1], sys.argv[2]
# HOME points at nothing so that only PATH is searched, not a uv store
interpreter = get_interpreter(
    "python3.11",
    cache=DiskCache(Path(cache_dir)),
    env={"PATH": search_path, "HOME": "/nonexistent"},
)
print(interpreter)

Output on 004294957ed227fe7e52b2864952ba09fd28b220:

--- first python3.11 on PATH is candidate_not_json.sh
it was probed as:
/tmp/tmpyp8sb7ev/bin/python3.11 /src/src/python_discovery/_py_info_collect.py 400df860d9380ee66ed61f844a4c0bb6 07980f9b5904657cb545f433f35b61bd
/tmp/tmpyp8sb7ev/bin/python3.11 /src/src/python_discovery/_py_info_collect.py be0bf87598b66fc12bb7f7f74d19a749 9988ae8be0ec755644196dc6a08da056
get_interpreter() returned: PathPythonInfo(spec=CPython3.11.2.final.0-64-x86_64, exe=/usr/bin/python3.11, platform=linux, version='3.11.2 (main, Oct  7 2026, 12:35:07) [GCC 12.2.0]', encoding_fs_io=utf-8-utf-8)
embedder stderr, last lines:
subprocess /tmp/tmpyp8sb7ev/bin/python3.11 returned invalid JSON; raw stdout 9 chars, start cookie missing, end cookie missing, parsed output 9 chars: 'not json\n'
subprocess /tmp/tmpyp8sb7ev/bin/python3.11 returned invalid JSON; raw stdout 9 chars, start cookie missing, end cookie missing, parsed output 9 chars: 'not json\n'

--- first python3.11 on PATH is candidate_empty_object.sh
it was probed as:
/tmp/tmpszcdx61j/bin/python3.11 /src/src/python_discovery/_py_info_collect.py 696e53e8a794de383f364fa5a5ebe90f ce1031a471478e5711ab333d27b35a9a
get_interpreter() returned: nothing
embedder stderr, last lines:
  File "/src/src/python_discovery/_py_info.py", line 448, in from_dict
    data["version_info"] = VersionInfo(**data["version_info"])  # restore this to a named tuple structure
                                         ~~~~^^^^^^^^^^^^^^^^
KeyError: 'version_info'

--- first python3.11 on PATH is candidate_version_info_only.sh
it was probed as:
/tmp/tmpoawmdrnw/bin/python3.11 /src/src/python_discovery/_py_info_collect.py 7a371205525a10d3aa610546160e4d6e 7aeb446fa06d10314d9e6aa4fc6810f5
get_interpreter() returned: nothing
embedder stderr, last lines:
  File "/src/src/python_discovery/_py_info.py", line 427, in _from_exe_with_args
    _LOGGER.info("ignore %s due cannot resolve system due to %r", proposed.original_executable, exception)
                                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'PathPythonInfo' object has no attribute 'original_executable'

BUG

Expected: Discovery should reject that candidate and continue searching.

Impact

Reproducibility is moderate because the setup requires a specific candidate or configuration. Exposure is niche: a non-Python candidate must print valid JSON and exit successfully. The reproduced effect stays within the caller's configured cache, search path, or interpreter.

Credits

Reported by Anthropic's OSS Scanner: ANT-2026-T9IA1FTF.

Activity

  1. added a commit that references this issue on Nov 3, 2026
    75e541f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions