Skip to content

Concurrent deallocation of threads while calling PyEval_SetTrace #132296

Description

@devdanzin

Crash report

What happened?

The following code will cause the interpreter to abort in a no-gil build:

import threading

for x in range(1000):
    threading._start_joinable_thread(lambda: None)
    try:
        threading.settrace_all_threads(())
    except Exception:
        pass

Abort backtrace:

python: Python/legacy_tracing.c:435: is_tstate_valid: Assertion `!_PyMem_IsPtrFreed(tstate)' failed.
Thread 1 "python" received signal SIGABRT, Aborted.

__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350580032) at ./nptl/pthread_kill.c:44
44      ./nptl/pthread_kill.c: No such file or directory.

#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350580032) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737350580032) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737350580032, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff7ce0476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7cc67f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff7cc671b in __assert_fail_base (fmt=0x7ffff7e7b130 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=0x555555b53aa9 "!_PyMem_IsPtrFreed(tstate)", file=0x555555b53a91 "Python/legacy_tracing.c", line=435,
    function=<optimized out>) at ./assert/assert.c:94
#6  0x00007ffff7cd7e96 in __GI___assert_fail (assertion=assertion@entry=0x555555b53aa9 "!_PyMem_IsPtrFreed(tstate)",
    file=file@entry=0x555555b53a91 "Python/legacy_tracing.c", line=line@entry=435,
    function=function@entry=0x555555b53ed0 <__PRETTY_FUNCTION__.18> "is_tstate_valid") at ./assert/assert.c:103
#7  0x000055555594a667 in is_tstate_valid (tstate=tstate@entry=0xdddddddddddddddd) at Python/legacy_tracing.c:435
#8  0x000055555594c0c2 in _PyEval_SetTrace (tstate=tstate@entry=0xdddddddddddddddd,
    func=func@entry=0x5555559984b0 <trace_trampoline>, arg=arg@entry=()) at Python/legacy_tracing.c:596
#9  0x000055555584cd32 in PyEval_SetTraceAllThreads (func=0x5555559984b0 <trace_trampoline>, arg=()) at Python/ceval.c:2473
#10 0x000055555599698e in sys__settraceallthreads (module=<optimized out>, arg=<optimized out>) at ./Python/sysmodule.c:1187
#11 0x0000555555859272 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555d8a858 <_PyRuntime+361432>, frame=0x7ffff7fb0098,
    frame@entry=0x7ffff7fb0020, throwflag=throwflag@entry=0) at Python/generated_cases.c.h:2271
#12 0x0000555555888b6d in _PyEval_EvalFrame (throwflag=0, frame=0x7ffff7fb0020, tstate=0x555555d8a858 <_PyRuntime+361432>)
    at ./Include/internal/pycore_ceval.h:119

It's also possible to sometimes segfault the interpreter with similar code, like:

import threading

threading._start_joinable_thread(lambda: None)
threading._start_joinable_thread(lambda: None)
threading._start_joinable_thread(lambda: None)
threading._start_joinable_thread(lambda: None)
for x in range(100):
    try:
        threading.settrace_all_threads(())
    except Exception:
        pass

Segfault backtrace:

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x000055555594b3fb in setup_tracing (tstate=tstate@entry=0x555555e29760, func=func@entry=0x5555559984b0 <trace_trampoline>, arg=arg@entry=(), old_traceobj=old_traceobj@entry=0x7fffffffd380) at Python/legacy_tracing.c:588
588         tstate->interp->sys_tracing_threads += delta;
(gdb) bt
#0  0x000055555594b3fb in setup_tracing (tstate=tstate@entry=0x555555e29760, func=func@entry=0x5555559984b0 <trace_trampoline>,
    arg=arg@entry=(), old_traceobj=old_traceobj@entry=0x7fffffffd380) at Python/legacy_tracing.c:588
#1  0x000055555594c141 in _PyEval_SetTrace (tstate=tstate@entry=0x555555e29760, func=func@entry=0x5555559984b0 <trace_trampoline>,
    arg=arg@entry=()) at Python/legacy_tracing.c:610
#2  0x000055555584cd32 in PyEval_SetTraceAllThreads (func=0x5555559984b0 <trace_trampoline>, arg=()) at Python/ceval.c:2473
#3  0x000055555599698e in sys__settraceallthreads (module=<optimized out>, arg=<optimized out>) at ./Python/sysmodule.c:1187
#4  0x00005555557110b6 in cfunction_vectorcall_O (
    func=<built-in method _settraceallthreads of module object at remote 0x20000259930>, args=<optimized out>,
    nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:537
#5  0x00005555556817dd in _PyObject_VectorcallTstate (tstate=0x555555d8a858 <_PyRuntime+361432>,
    callable=<built-in method _settraceallthreads of module object at remote 0x20000259930>, args=0x7fffffffd708,
    nargsf=9223372036854775809, kwnames=0x0) at ./Include/internal/pycore_call.h:169
#6  0x00005555556818fc in PyObject_Vectorcall (
    callable=callable@entry=<built-in method _settraceallthreads of module object at remote 0x20000259930>,
    args=args@entry=0x7fffffffd708, nargsf=<optimized out>, kwnames=kwnames@entry=0x0) at Objects/call.c:327
#7  0x000055555585575c in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555d8a858 <_PyRuntime+361432>, frame=0x7ffff7fb0098,
    frame@entry=0x7ffff7fb0020, throwflag=throwflag@entry=0) at Python/generated_cases.c.h:1434
#8  0x0000555555888b6d in _PyEval_EvalFrame (throwflag=0, frame=0x7ffff7fb0020, tstate=0x555555d8a858 <_PyRuntime+361432>)
    at ./Include/internal/pycore_ceval.h:119

Found using fusil by @vstinner.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a6+ experimental free-threading build (heads/main:0f04f2456a2, Apr 8 2025, 11:49:24) [GCC 11.4.0]

Linked PRs

Activity

  1. added
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Apr 9, 2025
  2. ZeroIntensity commented on Apr 9, 2025

    @ZeroIntensity
    Member

    The issue is that PyEval_SetTraceAllThreads uses another thread's state without holding the runtime lock, so it's free to get deallocated in the middle of execution. Normally, we'd either stop-the-world or grab the runtime lock, but unfortunately, PyEval_SetTrace is re-entrant, so neither of those work.

  3. changed the title [-]Abort and segfault from `threading.settrace_all_threads` in free-threading build[/-] [+]Concurrent deallocation of threads while calling `PyEval_SetTrace`[/+] on Apr 9, 2025
  4. kumaraditya303 commented on Aug 1, 2025

    @kumaraditya303
    Contributor

    This has the same underlying issue as #132817

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions