Function args sometimes not decref:ed after return

Bug report

Bug description:

I have an application written in C and Python, which implements threading using pthreads for performance-critical C code, and occasionally runs python code from these threads, protected by the GIL. I have somewhat complex reference counting logic which I unit test using weakref.Ref. These tests work flawlessly in 3.13 and earlier, but started to break in 3.14 (specifically 3.14.2, compiled with free threading disabled, linux/SLES12). My test looks something like:

from weakref import Ref
import gc
# my_c_module is implemented using the Python C API
from my_c_module import call_in_thread, f

class Data: pass

data = Data()
ref = Ref(data)
def g():
    f(data)

call_in_thread(g)

assert ref() is not None
del data
gc.collect()
assert ref() is None

By inspecting reference counts it seems that data sometimes gets an extra reference while returning from f: Py_REFCNT always returns 4 in the end of f, but right after returning into g, sys.getrefcount returns either 3 or 4. With 3.14 the test fails maybe 75% of the time, and with 3.10..3.13 it has never failed over thousands of runs. Unfortunately, I have not yet been able to reproduce it in a toy environment.

Playing around a bit, it seems to only happen when the variable is referenced from different pthreads, and only when a value is passed as an argument to a function implemented in C.

I also dug a bit with GDB, seems like the difference between the failing and succeeding path is this code path in _Py_Dealloc, which seems to defer deallocation of the args tuple, thus inhibiting arg decref:

    if (margin < 2 && gc_flag) {
        _PyTrash_thread_deposit_object(tstate, (PyObject *)op);
        return;
    }

It seems that these objects are leaked forever; that tuple object seems to remain even after I do things like call_in_thread(lambda: gc.collect()).

Any ideas? In particular, do you have any ideas on how to create a reproducer I can share? (the full application is in part proprietary, so cannot share that)

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Few comments:

Is call_in_thread blocking, or not? Can we be sure that after this call no other thread will use the data?
In the normal case you don't need to call gc.collect. del data is enough to "deallocate" the data
_PyTrash_thread_deposit_object means that you have "too deep" callstack. This means that the VM doesn't have enough space to free up the object, so it needs to postpone it. Is this your case?

call_in_thread is blocking. In the real application it's somewhat more complex, something like:

initialize()
run_after_steps(1, g)
run_steps(2)

So the thread will be done with the data, and apart from that only the initial Python thread will use the data.

Understood. My original test did not collect(); I added the call now just in case.
The call stack is very shallow, by all metrics I can find. While playing around, traceback.print_stack never printed more than two frames, and bt in gdb never printed more than 20 frames.

Thanks!
Could you please test with 3.14.0 and 3.14.1 versions?

I can reproduce it on 3.14.1. My application does not work with 3.14.0 because of #139653 (summary here)

git blame points at commits like 44e4c47, from July 2025. If you think it's reasonably easy to revert commits of this age, and if this is a reasonable suspicion, then I could try to revert it on top of 3.14.2.

FWIW, the margin returned by _Py_RecursionLimit_GetMargin is -3, perhaps that's a bad sign? From gdb:

Thread 3 "execution" hit Breakpoint 5, _Py_RecursionLimit_GetMargin (tstate=0x7fffe8001060)
    at ./Include/internal/pycore_pystate.h:330
warning: Source file is more recent than executable.
330	    return Py_ARITHMETIC_RIGHT_SHIFT(intptr_t, here_addr - (intptr_t)_tstate->c_stack_soft_limit, _PyOS_STACK_MARGIN_SHIFT);
(gdb) list
325	{
326	    _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
327	    assert(_tstate->c_stack_hard_limit != 0);
328	    intptr_t here_addr = _Py_get_machine_stack_pointer();
329	#if _Py_STACK_GROWS_DOWN
330	    return Py_ARITHMETIC_RIGHT_SHIFT(intptr_t, here_addr - (intptr_t)_tstate->c_stack_soft_limit, _PyOS_STACK_MARGIN_SHIFT);
331	#else
332	    return Py_ARITHMETIC_RIGHT_SHIFT(intptr_t, (intptr_t)_tstate->c_stack_soft_limit - here_addr, _PyOS_STACK_MARGIN_SHIFT);
333	#endif
334	}
(gdb) p/x here_addr
$3 = 0x7ffff2b76980
(gdb) p/x_tstate->c_stack_soft_limit
$7 = 0x7ffff2b81000
(gdb) n
_Py_Dealloc (op=op@entry=0x7ffff4aa4a00) at Objects/object.c:3050
warning: Source file is more recent than executable.
3050	    if (margin < 2 && gc_flag) {
(gdb) p margin
$8 = -3
(gdb)

As I previously mentioned, the test occasionally passes. I added debug some prints for margin. It's consistently -3 when it fails, and somewhere around 5000 (+-200) when it succeeds.

@markshannon

FWIW, I tried to blindly revert that margin check (I resolved a few merge conflicts by guessing):

https://github.com/mandolaerik/cpython/tree/pr/revert-gh-124715-move-trashcan-mechanism-into-py_dealloc-gh-132280

This revert makes my test pass consistently.

ping @markshannon ?

Our application is using fibers, i.e. stacks are moving between threads sometimes. I can see that _Py_RecursionLimit_GetMargin looks at c_stack_hard_limit from PyThreadState, maybe that member needs to be adjusted somehow when we move stacks between threads?

@vstinner

Maybe this will be useful #139653, capi-workgroup/decisions#81

cc @vstinner

I also dug a bit with GDB, seems like the difference between the failing and succeeding path is this code path in _Py_Dealloc, which seems to defer deallocation of the args tuple, thus inhibiting arg decref

Extract of Py_Dealloc():

    intptr_t margin = _Py_RecursionLimit_GetMargin(tstate);
    if (margin < 2 && gc_flag) {
        _PyTrash_thread_deposit_object(tstate, (PyObject *)op);
        return;
    }

_PyTrash_thread_deposit_object() is called if margin is smaller than 2. According to what you wrote, your call stack is not deep, and so it should not happen.

I can reproduce it on 3.14.1. My application does not work with 3.14.0 because of #139653 (summary here)

Aha, your C extension changes the stack memory. In this case, you should call PyUnstable_ThreadState_SetStack() with the new stack address and size. Can you modify your C extension to do that?

Thanks! PyUnstable_ThreadState_SetStackProtection was indeed what I missed. Does not solve all my test failures but most of them.

This issue can be closed from my perspective. It was hard to track down so if it's easy and cheap to add an explicit assertion failure with a nice message, then that could be useful for others. E.g., I suppose margin < 0 (see issue description) would indicate something is bad with the stack, although there are probably better places to insert the check.

I'm going to close it. If you want to improve the assertion message, you can do so in a separate issue and PR (but you can link this issue as well)

Navigation Menu

Repository navigation

Function args sometimes not decref:ed after return #144165

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Function args sometimes not decref:ed after return #144165

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Activity

sergey-miryanov commented on Jan 22, 2026

mandolaerik commented on Jan 22, 2026

sergey-miryanov commented on Jan 22, 2026

mandolaerik commented on Jan 23, 2026

mandolaerik commented on Jan 23, 2026

mandolaerik commented on Jan 23, 2026

mandolaerik commented on Jan 23, 2026

mandolaerik commented on Jan 23, 2026

sergey-miryanov commented on Jan 23, 2026

vstinner commented on Jan 23, 2026

mandolaerik commented on Jan 27, 2026

picnixz commented on Jan 27, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions