gh-154524: Fix race `array.array` export counter in free threading build by LindaSummer · Pull Request #157759 · python/cpython

LindaSummer · 2026-09-18T15:18:59Z

Proposed Changes

Guard arrayobject.ob_exports with atomic in free-threading build to fix racing during memoryview construction and release.

Root cause

During investigation array.array in #154524 , I find that the arrayobject.ob_exports is not protected in array_buffer_getbuf and array_buffer_relbuf.

Here is my test case in TSAN free-threading build on main.

from array import array
import threading

arr = array('i', [1, 2, 3, 4, 5])

ITER_TIMES = 1000
THREAD_NUMS = 4
def incre_export(arr: array):
    for _ in range(ITER_TIMES):
        view = memoryview(arr)
        view.release()

threads = [threading.Thread(target=incre_export, args=(arr,)) for _ in range(THREAD_NUMS)]
for t in threads:
    t.start()
for t in threads:
    t.join()

Here is the TSAN report of this case.

==================
WARNING: ThreadSanitizer: data race (pid=1161201)
  Write of size 8 at 0x7f799e715b88 by thread T1:
    #0 array_buffer_relbuf /home/someuser/projects/cpython/cpython-cdata-154525/./Modules/arraymodule.c:2921:21 (array.cpython-316t-x86_64-linux-gnu.so+0xc7d6)
    #1 PyBuffer_Release /home/someuser/projects/cpython/cpython-cdata-154525/Objects/abstract.c:825:9 (python+0x1eaf16)
    #2 mbuf_release /home/someuser/projects/cpython/cpython-cdata-154525/Objects/memoryobject.c:116:5 (python+0x2df7ef)
    #3 _memory_release /home/someuser/projects/cpython/cpython-cdata-154525/Objects/memoryobject.c:1118:9 (python+0x2df7ef)
    #4 memoryview_release_impl /home/someuser/projects/cpython/cpython-cdata-154525/Objects/memoryobject.c:1134:9 (python+0x2df7ef)
    #5 memoryview_release /home/someuser/projects/cpython/cpython-cdata-154525/Objects/clinic/memoryobject.c.h:147:12 (python+0x2df7ef)
    #6 _PyEval_EvalFrameDefault /home/someuser/projects/cpython/cpython-cdata-154525/Python/generated_cases.c.h:4330:35 (python+0x46b71d)
    #7 _PyEval_EvalFrame /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_ceval.h:122:16 (python+0x45dbc7)
    #8 _PyEval_Vector /home/someuser/projects/cpython/cpython-cdata-154525/Python/ceval.c:2174:12 (python+0x45dbc7)
    #9 _PyFunction_Vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c (python+0x227e77)
    #10 _PyObject_VectorcallTstate /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_call.h:144:11 (python+0x229bb4)
    #11 _PyObject_VectorcallPrepend /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:855:20 (python+0x229bb4)
    #12 method_vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/classobject.c:55:12 (python+0x22d606)
    #13 _PyObject_VectorcallTstate /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_call.h:144:11 (python+0x4bb839)
    #14 context_run /home/someuser/projects/cpython/cpython-cdata-154525/Python/context.c:802:29 (python+0x4bb839)
    #15 method_vectorcall_FASTCALL_KEYWORDS /home/someuser/projects/cpython/cpython-cdata-154525/Objects/descrobject.c:421:24 (python+0x242b90)
    #16 _PyObject_VectorcallTstate /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_call.h:144:11 (python+0x2277ac)
    #17 PyObject_Vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:327:12 (python+0x2277ac)
    #18 _Py_VectorCallInstrumentation_StackRefSteal /home/someuser/projects/cpython/cpython-cdata-154525/Python/ceval.c:768:11 (python+0x45e934)
    #19 _PyEval_EvalFrameDefault /home/someuser/projects/cpython/cpython-cdata-154525/Python/generated_cases.c.h:1906:35 (python+0x46511a)
    #20 _PyEval_EvalFrame /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_ceval.h:122:16 (python+0x45dbc7)
    #21 _PyEval_Vector /home/someuser/projects/cpython/cpython-cdata-154525/Python/ceval.c:2174:12 (python+0x45dbc7)
    #22 _PyFunction_Vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c (python+0x227e77)
    #23 _PyObject_VectorcallTstate /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_call.h:144:11 (python+0x229bb4)
    #24 _PyObject_VectorcallPrepend /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:855:20 (python+0x229bb4)
    #25 method_vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/classobject.c:55:12 (python+0x22d606)
    #26 _PyVectorcall_Call /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:273:16 (python+0x227ac6)
    #27 _PyObject_Call /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:348:16 (python+0x227ac6)
    #28 PyObject_Call /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:373:12 (python+0x227b3b)
    #29 thread_run /home/someuser/projects/cpython/cpython-cdata-154525/./Modules/_threadmodule.c:388:21 (python+0x65e9bc)
    #30 pythread_wrapper /home/someuser/projects/cpython/cpython-cdata-154525/Python/thread_pthread.h:236:5 (python+0x57caef)

  Previous write of size 8 at 0x7f799e715b88 by thread T2:
    #0 array_buffer_getbuf /home/someuser/projects/cpython/cpython-cdata-154525/./Modules/arraymodule.c:2913:21 (array.cpython-316t-x86_64-linux-gnu.so+0xc794)
    #1 PyObject_GetBuffer /home/someuser/projects/cpython/cpython-cdata-154525/Objects/abstract.c:455:15 (python+0x1eb5b5)
    #2 _PyManagedBuffer_FromObject /home/someuser/projects/cpython/cpython-cdata-154525/Objects/memoryobject.c:97:9 (python+0x2d7c56)
    #3 PyMemoryView_FromObjectAndFlags /home/someuser/projects/cpython/cpython-cdata-154525/Objects/memoryobject.c:813:42 (python+0x2d7c56)
    #4 PyMemoryView_FromObject /home/someuser/projects/cpython/cpython-cdata-154525/Objects/memoryobject.c:856:12 (python+0x2db190)
    #5 memoryview_impl /home/someuser/projects/cpython/cpython-cdata-154525/Objects/memoryobject.c:1017:12 (python+0x2db190)
    #6 memoryview /home/someuser/projects/cpython/cpython-cdata-154525/Objects/clinic/memoryobject.c.h:63:20 (python+0x2db190)
    #7 type_call /home/someuser/projects/cpython/cpython-cdata-154525/Objects/typeobject.c:2442:11 (python+0x3513bf)
    #8 _PyObject_MakeTpCall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:242:18 (python+0x226963)
    #9 _PyObject_VectorcallTstate /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_call.h:142:16 (python+0x227874)
    #10 PyObject_Vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:327:12 (python+0x227874)
    #11 _Py_VectorCallInstrumentation_StackRefSteal /home/someuser/projects/cpython/cpython-cdata-154525/Python/ceval.c:768:11 (python+0x45e934)
    #12 _PyEval_EvalFrameDefault /home/someuser/projects/cpython/cpython-cdata-154525/Python/generated_cases.c.h:1906:35 (python+0x46511a)
    #13 _PyEval_EvalFrame /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_ceval.h:122:16 (python+0x45dbc7)
    #14 _PyEval_Vector /home/someuser/projects/cpython/cpython-cdata-154525/Python/ceval.c:2174:12 (python+0x45dbc7)
    #15 _PyFunction_Vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c (python+0x227e77)
    #16 _PyObject_VectorcallTstate /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_call.h:144:11 (python+0x229bb4)
    #17 _PyObject_VectorcallPrepend /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:855:20 (python+0x229bb4)
    #18 method_vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/classobject.c:55:12 (python+0x22d606)
    #19 _PyObject_VectorcallTstate /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_call.h:144:11 (python+0x4bb839)
    #20 context_run /home/someuser/projects/cpython/cpython-cdata-154525/Python/context.c:802:29 (python+0x4bb839)
    #21 method_vectorcall_FASTCALL_KEYWORDS /home/someuser/projects/cpython/cpython-cdata-154525/Objects/descrobject.c:421:24 (python+0x242b90)
    #22 _PyObject_VectorcallTstate /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_call.h:144:11 (python+0x2277ac)
    #23 PyObject_Vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:327:12 (python+0x2277ac)
    #24 _Py_VectorCallInstrumentation_StackRefSteal /home/someuser/projects/cpython/cpython-cdata-154525/Python/ceval.c:768:11 (python+0x45e934)
    #25 _PyEval_EvalFrameDefault /home/someuser/projects/cpython/cpython-cdata-154525/Python/generated_cases.c.h:1906:35 (python+0x46511a)
    #26 _PyEval_EvalFrame /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_ceval.h:122:16 (python+0x45dbc7)
    #27 _PyEval_Vector /home/someuser/projects/cpython/cpython-cdata-154525/Python/ceval.c:2174:12 (python+0x45dbc7)
    #28 _PyFunction_Vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c (python+0x227e77)
    #29 _PyObject_VectorcallTstate /home/someuser/projects/cpython/cpython-cdata-154525/./Include/internal/pycore_call.h:144:11 (python+0x229bb4)
    #30 _PyObject_VectorcallPrepend /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:855:20 (python+0x229bb4)
    #31 method_vectorcall /home/someuser/projects/cpython/cpython-cdata-154525/Objects/classobject.c:55:12 (python+0x22d606)
    #32 _PyVectorcall_Call /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:273:16 (python+0x227ac6)
    #33 _PyObject_Call /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:348:16 (python+0x227ac6)
    #34 PyObject_Call /home/someuser/projects/cpython/cpython-cdata-154525/Objects/call.c:373:12 (python+0x227b3b)
    #35 thread_run /home/someuser/projects/cpython/cpython-cdata-154525/./Modules/_threadmodule.c:388:21 (python+0x65e9bc)
    #36 pythread_wrapper /home/someuser/projects/cpython/cpython-cdata-154525/Python/thread_pthread.h:236:5 (python+0x57caef)

So I added guard for all read and write patch for arrayobject.ob_exports to solve the problem.

From my understanding, this is a counter variable so an atomic guard should solve the problem.

Issue: Data race: ctypes PyCData_NewGetBuffer reads b_ptr without the critical section _ctypes_resize holds #154524

LindaSummer · 2026-09-18T15:23:38Z

Hi @devdanzin ,

Please help take a look when you have a chance.

By the way, for the CData resize problem, should we add an exporter counter and check it before resizing just like array.array's behavior?

Thanks very much and wish you a great day!

ZeroIntensity · 2026-09-18T16:16:39Z

For the record, there's an existing PR that makes array thread-safe: #130771

LindaSummer · 2026-09-20T23:18:44Z

For the record, there's an existing PR that makes array thread-safe: #130771

Hi @ZeroIntensity ,

Thanks very much for your record! ❤️
I will go through this implement and discussion and try to adapt into current patch.

ZeroIntensity · 2026-09-21T00:24:02Z

It would be very helpful if you could review that PR so we can land it instead of having to fix individual thread-safety problems like this.

LindaSummer · 2026-09-29T13:03:02Z

It would be very helpful if you could review that PR so we can land it instead of having to fix individual thread-safety problems like this.

Hi Peter,

Sorry that I was busy last week.
I will try to work on #130771 and update once there is any new progress. 😄

Same bug class as the memoryview hunk (#35). array_buffer_getbuf bumps ob_exports and array_buffer_relbuf drops it with a plain ++/-- and no critical section. A fiber that holds memoryview(arr) across a park and drops it on another hub has the old hub run the release (the biased- refcount merge deallocates the view where it was allocated) while the fiber takes the next view. A lost decrement pins the array for good (BufferError on resize); a lost increment lets it resize -- and free its items -- under a live view. Upstream too, unfixed on main and 3.15; python/cpython#157759 (gh-154524) proposes the same atomics. Both exec-home patches gain a Modules/arraymodule.c hunk: under Py_GIL_DISABLED && Py_TSTATE_EXEC_HOME the count is updated with _Py_atomic_add_ssize and the three "is it exporting?" checks read it relaxed; with the flag off every site expands back to stock. The patches also define _Py_ARRAY_EXPORTS_ATOMIC next to _Py_MV_EXPORTS_ATOMIC as an installed witness (arraymodule.c is not installed), and tools/ci/lib.sh checks both new witnesses. Guard: test_memory_array_view_survives_a_migration -- 64 ping-pong pairs at H=8, each holding a view of its own array across a park, batches to ~3000 OS-thread moves; after run() and a gc.collect() every array must resize with no view live and refuse to with one. It xfails, like the memoryview guard, on an interpreter without the witness. Measured (arm64 Darwin, no LTO/PGO): on the local interpreter built from #35's patches, 186-261 of 2112-2688 arrays per run lost an update (149-215 pinned, 37-49 resizable under a live view); on 3.14.4 built by tools/ci/build_patched_cpython.sh with these patches, none over 10 runs. CPython's test_array / test_buffer / test_memoryview pass on it (1155 tests). Both patches apply at -F0 to pristine 3.14.4 and 3.15.0rc2.

Same bug class as #35's memoryview fix. `array_buffer_getbuf` bumps `ob_exports` and `array_buffer_relbuf` drops it with a plain ++/-- and no critical section. A fiber holding `memoryview(arr)` across a park and dropping it on another hub has the old hub run the release (the biased-refcount merge deallocates the view where it was allocated) while the fiber takes the next view: a lost decrement pins the array for good (BufferError on resize), a lost increment lets it resize -- and free its items -- under a live view. Upstream too, unfixed on main and 3.15; python/cpython#157759 (filed under gh-154524) proposes the same fix. Both exec-home patches gain a Modules/arraymodule.c hunk: under `Py_GIL_DISABLED && Py_TSTATE_EXEC_HOME` the count is updated with `_Py_atomic_add_ssize`, and the three "is it exporting?" checks before a resize read it with acquire; with the flag off every site expands back to stock. It fixes lost updates only -- getbuf reads `ob_item` before counting the export, a window only a critical section closes and migration cannot reach. The object-header hunk defines `_Py_ARRAY_EXPORTS_ATOMIC` as an installed witness (arraymodule.c is not installed), and tools/ci/lib.sh checks both new witnesses. Guard: `test_memory_array_view_survives_a_migration` (64 ping-pong pairs at H=8, each holding a view of its own array across a park, to ~3000 OS-thread moves; every array must resize with no view live and refuse to with one; xfails on an interpreter without the witness). Unfixed: 186-261 of 2112-2688 arrays lost an update per run; with the hunk (3.14.4 built by tools/ci/build_patched_cpython.sh): none over 15 runs. CPython's test_array / test_buffer / test_memoryview pass on it; both patches apply at -F0 to pristine 3.14.4 and 3.15.0rc2.

LindaSummer added 2 commits September 18, 2026 10:15

fix array ob_exports racing in free threading

78bd5dd

add blurb

4046984

bedevere-app Bot mentioned this pull request Sep 18, 2026

Data race: ctypes PyCData_NewGetBuffer reads b_ptr without the critical section _ctypes_resize holds #154524

Open

bedevere-app Bot added the awaiting review label Sep 18, 2026

Merge branch 'main' into fix/array-export-race

2bac4b2

LindaSummer changed the title ~~gh-154524: Fix race array.arry export counter in free threading build~~ gh-154524: Fix race array.array export counter in free threading build Sep 18, 2026

johng mentioned this pull request Oct 1, 2026

exec-home: make array.array's buffer-export count atomic johng/stackweave#45

Merged

Navigation Menu

Repository navigation

gh-154524: Fix race `array.array` export counter in free threading build - #157759

gh-154524: Fix race `array.array` export counter in free threading build#157759
LindaSummer wants to merge 3 commits into
python:mainfrom
LindaSummer:fix/array-export-race

LindaSummer commented Sep 18, 2026 •

edited by bedevere-app Bot

Loading

Uh oh!

LindaSummer commented Sep 18, 2026

Uh oh!

ZeroIntensity commented Sep 18, 2026

Uh oh!

LindaSummer commented Sep 20, 2026 •

edited

Loading

Uh oh!

ZeroIntensity commented Sep 21, 2026

Uh oh!

LindaSummer commented Sep 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

LindaSummer commented Sep 18, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed Changes

Root cause

Uh oh!

LindaSummer commented Sep 18, 2026

Uh oh!

ZeroIntensity commented Sep 18, 2026

Uh oh!

LindaSummer commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZeroIntensity commented Sep 21, 2026

Uh oh!

LindaSummer commented Sep 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

LindaSummer commented Sep 18, 2026 •

edited by bedevere-app Bot

Loading

LindaSummer commented Sep 20, 2026 •

edited

Loading