Skip to content

Make TSAN tests pass with the GIL disabled in free-threaded builds #117657

Closed
#123697
Closed
Make TSAN tests pass with the GIL disabled in free-threaded builds#117657
#123697
Assignees
mpage
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-featureA feature request or enhancement

Description

@mpage
mpage
opened on Apr 8, 2024

Feature or enhancement

We need to make the TSAN tests pass with the GIL disabled before we can disable the GIL by default in free-threaded builds.

This should proceed in two phases:

  1. Add suppressions for existing TSAN warnings.
  2. Triage, fix, and remove the suppressions for the warnings enumerated in (1).

How to run the TSAN tests

  1. Build with TSAN:
    env CC=clang CXX=clang++ ./configure --disable-gil --with-thread-sanitizer --with-pydebug && make -j
  2. Run tests:
    env TSAN_OPTIONS=suppressions=<repo_root>/Tools/tsan/suppressions_free_threading.txt ./python -m test --tsan -j4

Working on a race

  1. Verify that the TSAN tests are passing using the steps from above.
  2. Pick a suppression from the section below and assign it to yourself (edit it and add your username next to it). Some of them may be related. If you find that other suppressions are related to the race you're working on please assign them to yourself or contact their owner if they're already assigned.
  3. Delete the suppression from <repo_root>/Tools/tsan/suppressions_free_threading.txt , run the TSAN tests, and verify that the race is reported by TSAN. You may need to comment out unrelated functions (notably, _PyEval_EvalFrameDefault) in order to reproduce the race.
  4. Fix the race.

Suppressions

### Tasks
- [ ] PyInterpreterState_ThreadHead
- [ ] PyThreadState_Clear
- [ ] PyThreadState_Next
- [ ] PyUnstable_InterpreterFrame_GetLine
- [ ] _PyFrame_GetCode
- [ ] _PyFrame_Initialize
- [ ] _PyObject_TryGetInstanceAttribute
- [ ] _Py_slot_tp_getattr_hook
- [x] _multiprocessing_SemLock_acquire_impl
- [x] _multiprocessing_SemLock_release_impl
- [ ] add_threadstate
- [ ] assign_version_tag
- [ ] dump_traceback
- [ ] fatal_error
- [ ] free_threadstate
- [ ] list_get_item_ref
- [x] new_reference
- [ ] pthread_create
- [ ] set_default_allocator_unlocked
- [ ] tstate_delete_common
- [ ] tstate_is_freed
- [ ] type_modified_unlocked
- [ ] write_thread_id

Linked PRs

Activity

  1. mpage
    added
    type-featureA feature request or enhancement
    topic-free-threading
    on Apr 8, 2024
  2. mpage
    self-assigned this
    on Apr 8, 2024
  3. colesbury
    added 4 commits that reference this issue on Apr 12, 2024

    Merge branch 'pythongh-117657-add-ft-tsan-suppressions' into nogil-in…

    73c8d7c

    gh-117657: Add TSAN suppressions for the free-threaded build (#117736)

    4783206

    gh-117657: Quiet TSAN warning about a data race between `start_the_wo…

    6e0b327

    gh-117657: Quiet more TSAN warnings due to incorrect modeling of comp…

    0d52383
  4. diegorusso
    added 3 commits that reference this issue on Apr 17, 2024

    pythongh-117657: Add TSAN suppressions for the free-threaded build (p…

    c267ff8

    pythongh-117657: Quiet TSAN warning about a data race between `start_…

    2b5a7e4

    pythongh-117657: Quiet more TSAN warnings due to incorrect modeling o…

    f8e2d81
  5. DinoV
    added 5 commits that reference this issue on Apr 17, 2024

    gh-117657: Fix data races in the method cache in free-threaded builds (…

    b6c62c7

    Merge branch 'main' into pythongh-117657-seq-lock

    778de8e

    gh-117657: Quiet erroneous TSAN reports of data races in `_PySeqLock` (…

    0d29302

    gh-117657: use relaxed loads for checking dict keys immortality (#118067

    1e4a4c4

    [gh-117657] _Py_MergeZeroLocalRefcount isn't loading ob_ref_shared wi…

    b45af00

  6. 275 remaining items

  7. colesbury
    added 3 commits that reference this issue on Mar 21, 2025

    pythongh-117657: Skip some tests when running with TSAN

    39cbaa3

    pythongh-117657: Fix TSAN data race in _PyEval_SetTrace assertion

    d035a90

    gh-117657: Fix TSAN data race in _PyEval_SetTrace assertion (gh-131561)

    0de5e0c
  8. miss-islington
    added a commit that references this issue on Mar 21, 2025

    pythongh-117657: Fix TSAN data race in _PyEval_SetTrace assertion (py…

    b38f60d
  9. colesbury
    added 2 commits that reference this issue on Mar 21, 2025

    [3.13] gh-117657: Fix TSAN data race in _PyEval_SetTrace assertion (g…

    94fbe97

    gh-117657: Skip some tests when running with TSAN (gh-131555)

    56d0f9a
  10. picnixz
    added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    on Mar 22, 2025
  11. colesbury
    added 2 commits that reference this issue on Mar 25, 2025

    pythongh-117657: Fix data race in `compare_unicode_unicode_threadsafe`

    31d8ac1

    gh-117657: Fix data race in `compare_unicode_unicode_threadsafe` (gh-…

    5abff69
  12. diegorusso
    added a commit that references this issue on Apr 1, 2025

    pythongh-117657: Fix data race in `compare_unicode_unicode_threadsafe` (

    0accaf5
  13. kumaraditya303
    added a commit that references this issue on Jul 8, 2025

    gh-117657: enable `test_capi` under TSAN (#136269)

    5193400
  14. AndPuQing
    added a commit that references this issue on Jul 11, 2025

    pythongh-117657: enable `test_capi` under TSAN (python#136269)

    b96cb1d
  15. Pranjal095
    added a commit that references this issue on Jul 12, 2025

    pythongh-117657: enable `test_capi` under TSAN (python#136269)

    67f6d60
  16. picnixz
    added a commit that references this issue on Jul 13, 2025

    pythongh-117657: enable `test_capi` under TSAN (python#136269)

    536f274

  17. nascheme commented on Jan 7, 2026

    @nascheme
    nascheme
    on Jan 7, 2026
    Member

    Great work everyone, this was a big one!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-featureA feature request or enhancement

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions