Skip to content

data race when resizing unicode objects #132070

Closed
Closed
data race when resizing unicode objects#132070
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtopic-unicodetype-bugAn unexpected behavior, bug, or error

Description

@kumaraditya303
kumaraditya303
opened on Apr 4, 2025

When running ctypes tests in parallel, the following data race is detected:

env TSAN_OPTIONS="suppressions=Tools/tsan/suppressions_free_threading.txt halt_on_error=1" ./python -m test test_ctypes --parallel-threads 4
==================
WARNING: ThreadSanitizer: data race (pid=19492)
  Atomic read of size 8 at 0x7f1b0a230680 by thread T1323:
    #0 _Py_atomic_load_uintptr_relaxed /home/realkumaraditya/cpython/./Include/cpython/pyatomic_gcc.h:375:10 (python+0x271055) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #1 _Py_IsOwnedByCurrentThread /home/realkumaraditya/cpython/./Include/object.h:254:12 (python+0x271055)
    #2 _Py_TryIncrefFast /home/realkumaraditya/cpython/./Include/internal/pycore_object.h:558:9 (python+0x271055)
    #3 _Py_TryIncrefCompare /home/realkumaraditya/cpython/./Include/internal/pycore_object.h:597:9 (python+0x271055)
    #4 compare_unicode_unicode_threadsafe /home/realkumaraditya/cpython/Objects/dictobject.c:1432:18 (python+0x271055)
    #5 do_lookup /home/realkumaraditya/cpython/Objects/dictobject.c:1013:23 (python+0x271055)
    #6 unicodekeys_lookup_unicode_threadsafe /home/realkumaraditya/cpython/Objects/dictobject.c:1449:12 (python+0x271055)
    #7 _Py_dict_lookup_threadsafe /home/realkumaraditya/cpython/Objects/dictobject.c:1504:18 (python+0x270757) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #8 _PyDict_GetItemRef_KnownHash /home/realkumaraditya/cpython/Objects/dictobject.c:2353:21 (python+0x272c0e) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #9 PyDict_GetItemRef /home/realkumaraditya/cpython/Objects/dictobject.c:2389:12 (python+0x272c0e)
    #10 cache_struct_converter /home/realkumaraditya/cpython/./Modules/_struct.c:2524:9 (_struct.cpython-314t-x86_64-linux-gnu.so+0x5338) (BuildId: cd844f32564aba7a2b18bdb1e258a78c397c8ebc)
    #11 calcsize /home/realkumaraditya/cpython/./Modules/clinic/_struct.c.h:266:10 (_struct.cpython-314t-x86_64-linux-gnu.so+0x4ac7) (BuildId: cd844f32564aba7a2b18bdb1e258a78c397c8ebc)
    #12 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:2225:35 (python+0x3fef50) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #13 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f7150) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #14 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f7150)
    #15 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1cdf) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #16 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f66b0) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #17 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:94:18 (python+0x1f66b0)
    #18 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f196f) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #19 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f196f)
    #20 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f19d5) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #21 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:2411:32 (python+0x3ff399) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #22 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f7150) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #23 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f7150)
    #24 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1cdf) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #25 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f65ff) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #26 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f65ff)
    #27 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f196f) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #28 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f196f)
    #29 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f19d5) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #30 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x5998d2) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #31 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f4157) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)

  Previous write of size 8 at 0x7f1b0a230680 by thread T1321:
    #0 __tsan_memcpy <null> (python+0xdd10f) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #1 _mi_memcpy /home/realkumaraditya/cpython/./Include/internal/mimalloc/mimalloc/internal.h:929:3 (python+0x2af9bf) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #2 _mi_heap_realloc_zero /home/realkumaraditya/cpython/Objects/mimalloc/alloc.c:745:7 (python+0x2af9bf)
    #3 mi_heap_realloc /home/realkumaraditya/cpython/Objects/mimalloc/alloc.c:753:10 (python+0x2cce30) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #4 _PyObject_MiRealloc /home/realkumaraditya/cpython/Objects/obmalloc.c:276:12 (python+0x2cce30)
    #5 PyObject_Realloc /home/realkumaraditya/cpython/Objects/obmalloc.c:1416:12 (python+0x2cfc12) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #6 resize_compact /home/realkumaraditya/cpython/Objects/unicodeobject.c:1178:31 (python+0x359915) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #7 _PyUnicodeWriter_Finish /home/realkumaraditya/cpython/Objects/unicodeobject.c:14157:16 (python+0x33162a) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #8 PyUnicode_Format /home/realkumaraditya/cpython/Objects/unicodeobject.c:15361:12 (python+0x35e728) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #9 unicode_mod /home/realkumaraditya/cpython/Objects/unicodeobject.c:14332:12 (python+0x36df71) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #10 binary_op1 /home/realkumaraditya/cpython/Objects/abstract.c:964:13 (python+0x1bdb83) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #11 binary_op /home/realkumaraditya/cpython/Objects/abstract.c:1003:24 (python+0x1be2a8) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #12 PyNumber_Remainder /home/realkumaraditya/cpython/Objects/abstract.c:1187:1 (python+0x1be2a8)
    #13 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:62:35 (python+0x3f7ae5) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #14 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f7150) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #15 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f7150)
    #16 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1cdf) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #17 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f66b0) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #18 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:94:18 (python+0x1f66b0)
    #19 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f196f) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #20 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f196f)
    #21 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f19d5) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #22 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:2411:32 (python+0x3ff399) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #23 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f7150) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #24 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f7150)
    #25 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1cdf) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #26 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f65ff) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #27 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f65ff)
    #28 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f196f) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #29 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f196f)
    #30 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f19d5) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #31 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x5998d2) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)
    #32 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f4157) (BuildId: 27b79e16bbd11b196a3f4a8232b4b09ecf4e18ee)

Linked PRs

Activity

  1. kumaraditya303
    added
    topic-free-threading
    on Apr 4, 2025
  2. picnixz
    added
    type-bugAn unexpected behavior, bug, or error
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    topic-unicode
    on Apr 4, 2025

  3. kumaraditya303 commented on Apr 4, 2025

    @kumaraditya303
    kumaraditya303
    on Apr 4, 2025
    ContributorAuthor

    I think in free-threading Py_REFCNT(op)==1 should be replaced by PyObject_IsUniquelyReferenced in such functions in unicodeobject.c:

    cpython/Objects/unicodeobject.c

    Lines 1197 to 1202 in a126cef

    static int
    resize_inplace(PyObject *unicode, Py_ssize_t length)
    {
    assert(!PyUnicode_IS_COMPACT(unicode));
    assert(Py_REFCNT(unicode) == 1);

  4. colesbury commented on Apr 4, 2025

    @colesbury
    colesbury
    on Apr 4, 2025
    Contributor

    We should leave the assertions as is. I don't think that's related to this.

    The race is because PyObject_Realloc() calls memcpy to fill in the new allocation, if the existing allocation can't be reused. The memcpy doesn't use relaxed atomic writes, so it can race with an one of the optimistic, non-locking reads.

    For now, we should just add a suppression.

  5. colesbury commented on Apr 4, 2025

    @colesbury
    colesbury
    on Apr 4, 2025
    Contributor

    We should change Py_REFCNT(op)==1 to PyObject_IsUniquelyReferenced for places where it controls behaviors, such as in if-statements, but the uses in assertions don't really matter.

  6. kumaraditya303
    added a commit that references this issue on Apr 16, 2025

    gh-132070: add `PyObject_Realloc` suppression in free-threading (#132468

    4b15d10

  7. godlygeek commented on Apr 27, 2025

    @godlygeek
    godlygeek
    on Apr 27, 2025
    Contributor

    Would it make sense to change the behavior of Py_REFCNT in free-threading so that Py_REFCNT(op)==1 remains a valid check? I think that due to TOCTOU the only thing an extension module can usefully do with Py_REFCNT even in GIL builds is check whether the recount of an object is or isn't 1. If you're no longer able to use Py_REFCNT to check that in FT builds, it seems like Py_REFCNT becomes totally useless, and it would be much more useful to do

    #define Py_REFCNT(x) (PyObject_IsUniquelyReferenced((x)) ? 1 : INT_MAX)

    or some other arbitrary large number.

  8. serhiy-storchaka commented on Apr 28, 2025

    @serhiy-storchaka
    serhiy-storchaka
    on Apr 28, 2025
    Member

    There are some places where Py_REFCNT is compared with 2 or 0.

  9. godlygeek commented on Apr 28, 2025

    @godlygeek
    godlygeek
    on Apr 28, 2025
    Contributor

    Sure, but those don't work in free threaded builds today, right? So my question is whether we can make it so that Py_REFCNT in an FT build returns something wrong but usually useful instead of something wrong and never useful.

  10. ZeroIntensity commented on Apr 28, 2025

    @ZeroIntensity
    ZeroIntensity
    on Apr 28, 2025
    Member

    Why couldn't it just be _PyObject_IsUniquelyReferenced(op) ? 1 : actual_refcnt(op)?

  11. colesbury commented on Apr 28, 2025

    @colesbury
    colesbury
    on Apr 28, 2025
    Contributor

    @ZeroIntensity - that's identical to actual_refcnt(op). _PyObject_IsUniquelyReferenced(1) == 1 implies Py_REFCNT(op) == 1. The inverse is not necessarily true.

    I agree with @serhiy-storchaka here. Py_REFCNT() is used for lots of things. Breaking it for a wide variety of use cases in order to try to make the _PyObject_IsUniquelyReferenced() use case more robust is not worth it.

  12. ZeroIntensity commented on Apr 28, 2025

    @ZeroIntensity
    ZeroIntensity
    on Apr 28, 2025
    Member

    The inverse is not necessarily true.

    Oh, that makes sense.

    • Is this documented anywhere? The docs do say that you can check for a reference count of 1. If that has problems with FT, we need to update it.
    • We should probably expose _PyObject_IsUniquelyReferenced as an unstable API for this then. Is there any current plan for that?
  13. corona10
    added a commit that references this issue on Apr 29, 2025

    gh-132070: Use _PyObject_IsUniquelyReferenced in unicodeobject (gh-13…

    75cbb8d
  14. miss-islington
    added a commit that references this issue on Apr 29, 2025

    pythongh-132070: Use _PyObject_IsUniquelyReferenced in unicodeobject (p…

    4f40bf2
  15. corona10
    added a commit that references this issue on Apr 29, 2025

    [3.13] gh-132070: Use _PyObject_IsUniquelyReferenced in unicodeobject (…

    7ffef8d

  16. corona10 commented on Apr 29, 2025

    @corona10
    corona10
    on Apr 29, 2025
    Member

    @ZeroIntensity Would you like to open a new issue to track adding unstable C API?

  17. corona10
    closed this as completedon Apr 29, 2025
  18. colesbury
    added 2 commits that reference this issue on Jan 5, 2026

    pythongh-132070: Fix PyObject_Realloc thread-safety in free threaded …

    a4c76a6

    gh-132070: Fix PyObject_Realloc thread-safety in free threaded Python (…

    98e55d7
  19. thunder-coding
    added a commit that references this issue on Feb 15, 2026

    pythongh-132070: Fix PyObject_Realloc thread-safety in free threaded …

    140f221
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtopic-unicodetype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions