Repository navigation
Sharing PySeqIter across threads use-after-frees the sequence under free-threading #156310
Description
Crash report
What happened?
On a free-threaded build, advancing a single, shared iterator over a legacy sequence (a type implementing __getitem__/sq_item but not __iter__/tp_iter) from multiple threads use-after-frees the underlying sequence object.
The defect is in the fallback iterator itself, so it affects pure-Python classes and C extension types alike. list (whose list_iterator was hardened for free-threading in #115605) is clean.
iter_iternext() in Objects/iterobject.c has no synchronization:
Lines 52 to 83 in 9cbd578
| static PyObject * | |
| iter_iternext(PyObject *iterator) | |
| { | |
| seqiterobject *it; | |
| PyObject *seq; | |
| PyObject *result; | |
| assert(PySeqIter_Check(iterator)); | |
| it = (seqiterobject *)iterator; | |
| seq = it->it_seq; | |
| if (seq == NULL) | |
| return NULL; | |
| if (it->it_index == PY_SSIZE_T_MAX) { | |
| PyErr_SetString(PyExc_OverflowError, | |
| "iter index too large"); | |
| return NULL; | |
| } | |
| result = PySequence_GetItem(seq, it->it_index); | |
| if (result != NULL) { | |
| it->it_index++; | |
| return result; | |
| } | |
| if (PyErr_ExceptionMatches(PyExc_IndexError) || | |
| PyErr_ExceptionMatches(PyExc_StopIteration)) | |
| { | |
| PyErr_Clear(); | |
| it->it_seq = NULL; | |
| Py_DECREF(seq); | |
| } | |
| return NULL; | |
| } |
The iterator owns exactly one reference to it_seq and releases it exactly once, on exhaustion. With the GIL that invariant holds because only one thread is ever inside tp_iternext. Without the GIL:
- Double DECREF on exhaustion. Several threads read the same non-
NULLit->it_seq, all observe out-of-bounds fromPySequence_GetItem, and each executes thePy_DECREF(seq). The single owned reference is released N times, so the sequence can be freed while other code is still holding refs to it. - Borrowed pointer outliving the object. Thread A is inside
PySequence_GetItem(seq, ...)(an arbitrarily long call) and thread B takes the exhaustion path and drops what may be the last reference. A then operates on freed memory.
(There's also a data race for index that #115605 fixed for lists with atomic stores, but per #124397, that's acceptable.)
calliter_iternext has the same defect: it_callable/it_sentinel are cleared with non-atomic Py_CLEAR on the exhaustion path and read without protection, so a shared callable_iterator can double-release and use-after-free them the same way.
There's prior art for this issue class in #154043, #154108, #154130 and the same fixes can be applied here.
Reproducer
import sys, threading
THREADS = 16
ROUNDS = 2000
class PySeq: # sq_item only -> iter() falls back to PySeqIter
def __init__(self, n):
self.n = n
def __len__(self):
return self.n
def __getitem__(self, i):
if i >= self.n:
raise IndexError(i)
return i
def drain(it, barrier):
barrier.wait()
while True:
try:
next(it)
except StopIteration:
return
except Exception:
return # torn it_index can walk off the end; benign
def hammer(make):
for _ in range(ROUNDS):
it = make()
barrier = threading.Barrier(THREADS)
ws = [
threading.Thread(target=drain, args=(it, barrier))
for _ in range(THREADS)
]
for w in ws:
w.start()
for w in ws:
w.join()
del it
seq = PySeq(4)
print("gil enabled:", sys._is_gil_enabled())
print("iter type:", type(iter(seq)).__name__)
before = sys.getrefcount(seq)
hammer(lambda: iter(seq))
after = sys.getrefcount(seq)
print(
f"before={before} after={after} -> {'CORRUPTED' if after != before else 'ok'}"
)Observed results
All on macOS / arm64 (Apple Silicon):
| Build | PySeq (PySeqIter) |
[0, 1, 2, 3] (list_iterator) |
|---|---|---|
main (3.16.0a0, free-threading debug build) |
SIGSEGV | clean, refcount ok |
| 3.14.6 free-threaded | SIGSEGV (3/3 runs) | clean (3/3 runs) |
| 3.13.8 free-threaded | SIGSEGV (3/3 runs) | not run |
| 3.14.6 GIL-enabled | clean, refcount ok | clean |
| 3.15.x free-threaded | refcount bad (on some runs; see below) | not run |
Depending on allocator timing the failure may show up as refcount corruption, type confusion in
unrelated code once the freed memory is reused, or as SIGSEGV, or nothing at all:
~/Desktop $ uv run --python=3.15t --isolated repro.py
gil enabled: False
iter type: iterator
~/Desktop $ uv run --python=3.15t --isolated repro.py
gil enabled: False
iter type: iterator
before=2 after=1152921504606846749 -> CORRUPTED
CPython versions tested on:
CPython main branch, 3.13, 3.14, 3.15
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
Python 3.16.0a0 free-threading build (heads/main:53d2e14a308, Aug 24 2026, 09:57:36) [Clang 17.0.0 (clang-1700.3.19.1)]
Activity
pythongh-156310: Make the iter() sequence fallback iterator safe in f…
gh-156310: Make the iter() sequence fallback iterator safe in free-th…
pythongh-156310: Make the iter() sequence fallback iterator safe in f…