Repository navigation
Candidate output can exhaust memory before the query timeout #192
Description
A candidate named like Python that writes endlessly to stdout makes interpreter discovery exhaust the caller's memory before the configured timeout.
Affected code
src/python_discovery/_cached_py_info.py:252 at the reported commit. The reproducer also returns BUG on current main 004294957ed227fe7e52b2864952ba09fd28b220.
Mechanism
The probe captures the candidate's output without a size limit while the candidate keeps writing.
Reproduction
Check out the reported commit or current main. Save the files below as named, then build and run:
docker build -t python-discovery-audit -f - . <<'DOCKERFILE'
FROM python:3.15-bookworm
RUN python -m venv /opt/venv
ENV PATH=/opt/venv/bin:$PATH
COPY . /src
WORKDIR /src
ENV SETUPTOOLS_SCM_PRETEND_VERSION=1.9.1
RUN pip install -e . --group test
DOCKERFILE
docker run --rm --network=none --memory=2g -u 0 -w /src \
-v "$PWD/repro:/tmp/repro:ro" -v "$PWD/repro.d:/tmp/repro.d:ro" \
python-discovery-audit /tmp/reprorepro
#!/usr/bin/env python3
import os
import resource
import shutil
import subprocess
import tempfile
import time
# setup: a PATH directory whose only entry is the launcher that never stops printing
path_dir = tempfile.mkdtemp()
launcher = shutil.copy("/tmp/repro.d/python3.11", path_dir)
os.chmod(launcher, 0o755)
# trigger: one get_interpreter("python3.11") over that PATH.
# The 1 GiB address-space cap stands in for the machine's RAM: it does not cause the growth,
# it ends it with a MemoryError after about a second instead of with the OOM killer.
ram = 1 << 30
started = time.monotonic()
caller = subprocess.run(
["/opt/venv/bin/python", "/tmp/repro.d/find_python.py", path_dir],
preexec_fn=lambda: resource.setrlimit(resource.RLIMIT_AS, (ram, ram)),
capture_output=True,
text=True,
timeout=60,
)
elapsed = time.monotonic() - started
peak_rss_mib = resource.getrusage(resource.RUSAGE_CHILDREN).ru_maxrss // 1024
# check
print("argv discovery spawned:", open(os.path.join(path_dir, "spawned.log")).read())
print("caller stdout:", caller.stdout)
# the traceback has no source lines and a garbled tail: the interpreter is out of memory while printing it
print("caller stderr:", caller.stderr)
print(f"caller exit status {caller.returncode} after {elapsed:.1f} s, peak RSS {peak_rss_mib} MiB")
probe_timeout = 15 # _DEFAULT_QUERY_TIMEOUT; the caller does not set PY_DISCOVERY_TIMEOUT
died_buffering_candidate_output = (
"in _run_subprocess\n" in caller.stderr and "in communicate\n" in caller.stderr and "MemoryError" in caller.stderr
)
print("MemoryError inside the probe's communicate():", died_buffering_candidate_output)
print(f"before the {probe_timeout} s probe timeout:", elapsed < probe_timeout)
print("BUG" if died_buffering_candidate_output and elapsed < probe_timeout else "OK")repro.d/find_python.py
"""An embedder (as virtualenv or tox would) asking python-discovery for a python3.11 from its PATH."""
import sys
import tempfile
from pathlib import Path
from python_discovery import DiskCache, get_interpreter
path_dir = sys.argv[1]
found = get_interpreter(
"python3.11",
cache=DiskCache(Path(tempfile.mkdtemp())),
env={"PATH": path_dir, "HOME": "/nonexistent"},
)
print("selected:", found)repro.d/python3.11
#!/bin/sh
# A launcher stuck in a retry loop: it records how it was invoked, then prints the same line until it is killed.
echo "$0 $*" >> "${0%/*}/spawned.log"
exec /usr/bin/yes "python3.11: waiting for licence server, retrying"Output on 004294957ed227fe7e52b2864952ba09fd28b220:
argv discovery spawned: /tmp/tmpz42ncoye/python3.11 /src/src/python_discovery/_py_info_collect.py ada92ba8614598fb2152c001b0da107e cf6868a84cf3a80de2d235541cf72489
caller stdout:
caller stderr: Traceback (most recent call last):
File "/tmp/repro.d/find_python.py", line 10, in <module>
File "/src/src/python_discovery/_discovery.py", line 52, in get_interpreter
File "/src/src/python_discovery/_discovery.py", line 135, in _find_interpreter
File "/src/src/python_discovery/_discovery.py", line 203, in propose_interpreters
File "/src/src/python_discovery/_discovery.py", line 261, in _propose_from_path
File "/src/src/python_discovery/_py_info.py", line 385, in from_exe
File "/src/src/python_discovery/_py_info.py", line 411, in _from_exe_with_args
File "/src/src/python_discovery/_cached_py_info.py", line 65, in from_exe
File "/src/src/python_discovery/_cached_py_info.py", line 87, in _get_from_cache
File "/src/src/python_discovery/_cached_py_info.py", line 127, in _get_via_file_cache
File "/src/src/python_discovery/_cached_py_info.py", line 252, in _run_subprocess
File "/usr/local/lib/python3.15/subprocess.py", line 1395, in communicate
File "/usr/local/lib/python3.15/subprocess.py", line 2412, in _communicate
File "/usr/local/lib/python3.15/subprocess.py", line 316, in _communicate_io_posix
object address : 0x7f3a3221dfc0
object refcount : 3
object type : 0x7f3a32b4a9e0
object type name: MemoryError
object repr : MemoryError()
lost sys.stderr
Exception ignored in atexit callback <function _resolve_lazy_imports at 0x7f3a3131e140>:
Traceback (most recent call last):
File "/opt/venv/lib/python3.15/site-packages/filelock/_api.py", line 1971, in _resolve_lazy_imports
File "<frozen importlib._bootstrap>", line 1353, in _find_and_load
File "<frozen importlib._bootstrap>", line 1249, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 857, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 747, in exec_module
File "<frozen importlib._bootstrap_external>", line 839, in get_code
File "<frozen importlib._bootstrap_external>", line 926, in get_data
MemoryError:
caller exit status 1 after 0.3 s, peak RSS 1017 MiB
MemoryError inside the probe's communicate(): True
before the 15 s probe timeout: True
BUG
Expected: The caller should bound captured output and skip the candidate.
Impact
Reproducibility is moderate because the setup requires a specific candidate or configuration. Exposure is niche: a broken wrapper or non-Python executable must appear under a Python name on the caller's PATH. The reproduced effect stays within the caller's configured cache, search path, or interpreter.
Credits
Reported by Anthropic's OSS Scanner: ANT-2026-FJZ4KEUG.
Activity