Repository navigation
Race in concurrent iteration over range iterators #129068
Description
Bug report
Bug description:
Concurrent iteration over range iterators (not range objects themselves) is not thread-safe, because range iterators are currently implemented as two pieces of state (the next value and the number of values left) that aren't updated atomically. ThreadSanitizer correctly identifies these races as well. See tests in PR #128637, and an early attempt at fixing the issues in 1533d1d.
This also affects bytecode specialisation (FOR_ITER_RANGE), although in PR #128798 we're only specialising if the iterator is uniquely referenced (and so we don't care about other threads).
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Activity
colesbury commentedon Mar 14, 2025
I wonder if we should apply the same technique to
fast_range_iterthat we do in the bytecode, where we don't need to care about other threads if_PyObject_IsUniquelyReferenced(iter)is true. We could fall back to a locking code path if that's not the case. (Apologies, if this is something you've already suggested previously).I'd want to wait for Matt's
LOAD_FASToptimizations to land first, because otherwise I think we'll slow down cases like:That pattern exists a lot in the pyperformance benchmarks and probably in some real world code too.
pythongh-129068: allow concurrent iteration over range iterators
pythongh-129068: Make range iterators thread-safe
pythongh-129068: Make range iterators thread-safe
Merge branch 'main' into pythongh-129068-rangeiter-next
Merge branch 'main' into pythongh-129068-rangeiter-next
gh-129068: Make range iterators thread-safe (gh-142886)
colesbury commentedon Dec 18, 2025
Fixed in main (3.15). I'm not planning to backport it to 3.14 unless someone asks for it.
pythongh-129068: Make range iterators thread-safe (pythongh-142886)