Segfault when cleaning up after a MemoryError with deeply nested objects

Crash report

What happened?

When in memory-constrained situations, the following code generates a segfault on main:

b = 9
while True:
    b = (b, None)

Clearly, b grows to be an unbounded nested tuple, which eventually exhausts the memory.
A MemoryError is correctly thrown, but during unwind, python tries to clean up the object, and this triggers a stack overflow.

Example stack

Process 369 stopped

Thread 1 "fuzz_python" stop reason = signal SIGSEGV
  frame #0: frame #0: 0x0000aaaaaea31648 fuzz_python`tuple_dealloc(self=0x0000ffff9fa045e0) at tupleobject.c:256
  frame #1: frame #1: 0x0000aaaaae9ccd54 fuzz_python`_Py_Dealloc(op=0x0000ffff9fa045e0) at object.c:3303:5
  frame #2: frame #2: 0x0000aaaaaea3182c fuzz_python`Py_DECREF(op=<unavailable>) at refcount.h:427:9 [inlined]
  frame #3: frame #3: 0x0000aaaaaea3180c fuzz_python`Py_XDECREF(op=<unavailable>) at refcount.h:520:9 [inlined]
  frame #4: frame #4: 0x0000aaaaaea31804 fuzz_python`tuple_dealloc(self=0x0000ffff9fa04630) at tupleobject.c:277:9
  frame #5: frame #5: 0x0000aaaaae9ccd54 fuzz_python`_Py_Dealloc(op=0x0000ffff9fa04630) at object.c:3303:5
  frame #6: frame #6: 0x0000aaaaaea3182c fuzz_python`Py_DECREF(op=<unavailable>) at refcount.h:427:9 [inlined]
  frame #7: frame #7: 0x0000aaaaaea3180c fuzz_python`Py_XDECREF(op=<unavailable>) at refcount.h:520:9 [inlined]
...
  frame #2709: frame #2709: 0x0000aaaaae9ccd54 fuzz_python`_Py_Dealloc(op=0x0000ffff9fa11cd0) at object.c:3303:5
  frame #2710: frame #2710: 0x0000aaaaaea3182c fuzz_python`Py_DECREF(op=<unavailable>) at refcount.h:427:9 [inlined]
  frame #2711: frame #2711: 0x0000aaaaaea3180c fuzz_python`Py_XDECREF(op=<unavailable>) at refcount.h:520:9 [inlined]
  frame #2712: frame #2712: 0x0000aaaaaea31804 fuzz_python`tuple_dealloc(self=0x0000ffff9fa11d20) at tupleobject.c:277:9
  frame #2713: frame #2713: 0x0000aaaaae9ccd54 fuzz_python`_Py_Dealloc(op=0x0000ffff9fa11d20) at object.c:3303:5
  frame #2714: frame #2714: 0x0000aaaaaea3182c fuzz_python`Py_DECREF(op=<unavailable>) at refcount.h:427:9 [inlined]
  frame #2715: frame #2715: 0x0000aaaaaea3180c fuzz_python`Py_XDECREF(op=<unavailable>) at refcount.h:520:9 [inlined]
  frame #2716: frame #2716: 0x0000aaaaaea31804 fuzz_python`tuple_dealloc(self=0x0000ffff9fa11d70) at tupleobject.c:277:9
  frame #2717: frame #2717: 0x0000aaaaae9ccd54 fuzz_python`_Py_Dealloc(op=0x0000ffff9fa11d70) at object.c:3303:5
  frame #2718: frame #2718: 0x0000aaaaaea3182c fuzz_python`Py_DECREF(op=<unavailable>) at refcount.h:427:9 [inlined]
  frame #2719: frame #2719: 0x0000aaaaaea3180c fuzz_python`Py_XDECREF(op=<unavailable>) at refcount.h:520:9 [inlined]
  frame #2720: frame #2720: 0x0000aaaaaea31804 fuzz_python`tuple_dealloc(self=0x0000ffff9fa11dc0) at tupleobject.c:277:9
  frame #2721: frame #2721: 0x0000aaaaae9ccd54 fuzz_python`_Py_Dealloc(op=0x0000ffff9fa11dc0) at object.c:3303:5
  frame #2722: frame #2722: 0x0000aaaaae98f290 fuzz_python`Py_DECREF(op=<unavailable>) at refcount.h:427:9 [inlined]
  frame #2723: frame #2723: 0x0000aaaaae98f26c fuzz_python`Py_XDECREF(op=<unavailable>) at refcount.h:520:9 [inlined]
  frame #2724: frame #2724: 0x0000aaaaae98f26c fuzz_python`dictkeys_decref(dk=0x0000ffffbe045210, use_qsbr=false) at dictobject.c:495:17
  frame #2725: frame #2725: 0x0000aaaaae98630c fuzz_python`dict_dealloc(self=0x0000ffffbe055f40) at dictobject.c:0
  frame #2726: frame #2726: 0x0000aaaaae9ccd54 fuzz_python`_Py_Dealloc(op=0x0000ffffbe055f40) at object.c:3303:5
  frame #2727: frame #2727: 0x0000aaaaae7ecb8c fuzz_python`Py_DECREF(op=0x0000ffffbe055f40) at refcount.h:427:9 [inlined]
  frame #2728: frame #2728: 0x0000aaaaae7ecb58 fuzz_python`main(argc=1, argv=0x0000fffff67f4238) at fuzz_python.c:226:17
  frame #2729: frame #2729: 0x0000ffffbe1e1f9c ld-musl-aarch64.so.1`libc_start_main_stage2(main=(fuzz_python`main at fuzz_python.c:71), argc=1, argv=0x0000fffff67f4238) at __libc_start_main.c:95:2

The following dockerfile reliably reproduces it for me:

Dockerfile

FROM alpine:latest

RUN apk add --no-cache \
    git build-base linux-headers openssl-dev zlib-dev bzip2-dev \
    xz-dev sqlite-dev readline-dev libffi-dev ncurses-dev gdbm-dev \
    tcl-dev tk-dev

WORKDIR /src
RUN git clone --depth=1 https://github.com/python/cpython.git
WORKDIR /src/cpython
RUN ./configure && make -j2

RUN cat > /tc.py <<'PY'
b = 9
while True:
    b = (b, None)
PY

RUN cat > /entrypoint.sh <<'SH'
#!/bin/sh

echo "Running test"
# 512 MiB virtual memory limit
ulimit -v 524288

/src/cpython/python /tc.py
echo "Python process exited with code $?"
SH

RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

Gives me:

>docker run --rm -it 29apr
Running test
Traceback (most recent call last):
  File "/tc.py", line 3, in <module>
object address  : 0xffffae809fc0
object refcount : 3
object type     : 0xaaaad32175a8
object type name: MemoryError
object repr     : MemoryError()
lost sys.stderr
Segmentation fault
Python process exited with code 139

ISTR that stack overflows from recursive objects were at least considered low priority, if not 'not worth fixing' before, but I can't recall where I read that, it was a while ago, so apologies if this isn't a useful report!

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.0a8+ (heads/main:d4eee16, Apr 29 2026, 11:03:26) [GCC 15.2.0]

Linked PRs

gh-149146: Add dealloc-depth counter fallback to trashcan trigger #151595

@markshannon

Bisect identified this as the change that introduced the segfault: @markshannon

147cd05 is the first bad commit
commit 147cd05
Author: Mark Shannon
Date: Wed Apr 17 11:08:05 2024 +0100

GH-117760: Streamline the trashcan mechanism (GH-117763)

Include/cpython/object.h | 47 ++++++++--------
Include/cpython/pystate.h | 7 +--
Include/internal/pycore_object.h | 1 -
Objects/object.c | 114 ++++++---------------------------------
Python/pystate.c | 2 +
5 files changed, 43 insertions(+), 128 deletions(-)
bisect found first bad commit

Hi — I'd like to take a look at this. Reproducing locally and reading through the tuple_dealloc path from the bisect. Will open a PR if I get somewhere.

The trace includes "musl": the musl C library. I recently modified Python to update runtime stack size checks on musl. See commit 9a61d1c (issue gh-151546).

Can you please check if you can still reproduce the issue on the current main branch?

@vstinner

@vstinner - Thank you! I've actually switched away from musl since this report, due to some memory issues, so that's good news.

My quick tests indicates this is still reproducible wthout musl:

echo 2 > /proc/sys/vm/overcommit_memory
~ #
~ # /pfm/py/bin/python3 /pfm/scratch/dist/gh149146.py
[  487.622390] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
[  487.622514] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
[  487.622595] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
[  487.622646] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
[  487.622759] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
[  487.622840] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
[  487.622878] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
[  487.622936] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
[  487.622987] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
[  487.623022] __vm_enough_memory: pid: 97, comm: python3, bytes: 1048576 not enough memory for the allocation
Exception ignored in the internal traceback machinery:
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1340, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1290, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1270, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1244, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1364, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1407, in _fill_cache
OSError: [Errno 12] Cannot allocate memory: '/pfm/scratch/dist'
Traceback (most recent call last):
  File "/pfm/scratch/dist/gh149146.py", line 3, in <module>
object address  : 0xfffff7c27700
object refcount : 3
object type     : 0xaaaaab78d7b0
object type name: MemoryError
object repr     : MemoryError()
lost sys.stderr
Segmentation fault
~ # /pfm/py/bin/python3 --version
Python 3.16.0a0
~ # ldd /pfm/py/bin/python3
	linux-vdso.so.1 (0x0000fffff7fff000)
	libm.so.6 => /usr/lib/libm.so.6 (0x0000fffff6e30000)
	libc.so.6 => /usr/lib/libc.so.6 (0x0000fffff6c60000)
	/lib/ld-linux-aarch64.so.1 => /usr/lib/ld-linux-aarch64.so.1 (0x0000fffff7fb0000)

that was with:

commit a064b323f4350305e7486c7b1090cf12b19e7738 (grafted, HEAD -> main, origin/main)
Author: Victor Stinner <vstinner@python.org>
Date:   Wed Jun 17 10:58:22 2026 +0200
gh-151496: Use process groups in test_dtrace (#151512)
...

I'll run again with latest main to verify

Yea, it looks to still be there:

~ # echo 2 > /proc/sys/vm/overcommit_memory
~ # lldb /pfm/py/bin/python3 /pfm/scratch/gh149146.py
[   28.116418] random: crng init done
(lldb) target create "/pfm/py/bin/python3"
Current executable set to '/pfm/py/bin/python3' (aarch64).
(lldb) settings set -- target.run-args  "/pfm/scratch/gh149146.py"
(lldb) run
Process 90 launched: '/pfm/py/bin/python3' (aarch64)
Exception ignored in the internal traceback machinery:
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1340, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1290, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1216, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1270, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1244, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1364, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1407, in _fill_cache
OSError: [Errno 12] Cannot allocate memory: '/pfm/scratch'
Traceback (most recent call last):
  File "/pfm/scratch/gh149146.py", line 3, in <module>
object address  : 0xfffff7c27700
object refcount : 3
object type     : 0xaaaaab78d870
object type name: MemoryError
object repr     : MemoryError()
lost sys.stderr
Process 90 stopped
* thread #1, name = 'python3', stop reason = signal SIGSEGV: address not mapped to object (fault address=0xfffffffddff0)
    frame #0: 0x0000aaaaaafebc58 python3`_Py_Dealloc(op=0x0000aaaaabdeac90) at object.c:3291
   3288	stack is shallower */
   3289	void
   3290	_Py_Dealloc(PyObject *op)
-> 3291	{
   3292	    PyTypeObject *type = Py_TYPE(op);
   3293	    unsigned long gc_flag = type->tp_flags & Py_TPFLAGS_HAVE_GC;
   3294	    destructor dealloc = type->tp_dealloc;

~ # /pfm/py/bin/python3 -VV
Python 3.16.0a0 (heads/main-dirty:12add38, Jun 18 2026, 11:18:38) [Clang 22.1.6 ]

Ok, I reproduced the issue on Fedora 44 (using glibc). I limited the virtual memory to ~1 GiB:

$ ulimit -v 1024000
$ gdb -args ./python b.py 
(gdb) run

gdb traceback:

#0  0x00000000005717f6 in _Py_RecursionLimit_GetMargin ()
#1  0x0000000000577dd0 in _Py_Dealloc ()
#2  0x00000000005ae871 in Py_DECREF ()
#3  0x00000000005ae8a1 in Py_XDECREF ()
#4  0x00000000005aff6c in tuple_dealloc ()
#5  0x0000000000577ed5 in _Py_Dealloc ()
#6  0x00000000005ae871 in Py_DECREF ()
#7  0x00000000005ae8a1 in Py_XDECREF ()
#8  0x00000000005aff6c in tuple_dealloc ()
#9  0x0000000000577ed5 in _Py_Dealloc ()
#10 0x00000000005ae871 in Py_DECREF ()
#11 0x00000000005ae8a1 in Py_XDECREF ()
#12 0x00000000005aff6c in tuple_dealloc ()
#13 0x0000000000577ed5 in _Py_Dealloc ()
#14 0x00000000005ae871 in Py_DECREF ()
#15 0x00000000005ae8a1 in Py_XDECREF ()
#16 0x00000000005aff6c in tuple_dealloc ()
#17 0x0000000000577ed5 in _Py_Dealloc ()
#18 0x00000000005ae871 in Py_DECREF ()
#19 0x00000000005ae8a1 in Py_XDECREF ()
#20 0x00000000005aff6c in tuple_dealloc ()
#21 0x0000000000577ed5 in _Py_Dealloc ()
#22 0x00000000005ae871 in Py_DECREF ()
...

Stack of the process:

$ grep stack /proc/1285237/maps
7ffffffc8000-7ffffffff000 rw-p 00000000 00:00 0                          [stack]

That's only 220 kiB, it's quite small! Glibc uses a stack size of 8 MiB and the stack is allocated on demand. Here, the stack cannot be allocated since we exhausted all the memory.

There isn't much that Python can do in this case: when the memory is exhausted, Python fails to allocate more stack and so it does crash with a stack overflow.

Another reproducer which limits the process memory to 1 GiB to not crash the whole system:

import resource

# deallocation the chain uses a lot of C stack memory
CHAIN_LENGTH = 1024 * 1024

# limit process memory to not crash the whole machine
max_memory = 1024 * 1024 * 1024
resource.setrlimit(resource.RLIMIT_AS, (max_memory, max_memory))

print("create long chain...")
long_chain = None
for _ in range(CHAIN_LENGTH):
    long_chain = (long_chain, None)

print("fill the process memory...")

memory = []
try:
    large = 1024 * 1024
    while True:
        memory.append(b'x' * large)
except MemoryError:
    pass

small = 1024
while True:
    memory.append(b'x' * small)

# at exit (on error), deallocating long_chain will use a lot of C stack memory
# ... and does crash!

@bhuvi27

Aah, I hadn't realised that this was that stack resize failure issue (thanks!) I'd already separately raised #150722 to cover that, and agree it's a hard one to detect.

@bhuvi27 has a PR, but I have no idea if that approach is sensible or not.

Oh, when I ran my reproducer script, I found yet another bug related to error handling of MemoryError: issue #152090.

So the problem is that the system/process memory is exhausted, and so when the Linux kernel tries to extend the process stack memory, it fails with a segfault since there is no more available memory.

There is nothing Python can do in this case. There is no generic and safe mechanism to catch the segfault and report it as a regular Python exception. Catching SIGSEGV signal is dangerous.

I close the issue as "won't fix".

Navigation Menu

Repository navigation

Segfault when cleaning up after a MemoryError with deeply nested objects #149146

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Segfault when cleaning up after a MemoryError with deeply nested objects #149146

Description

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Activity

stestagg commented on Apr 29, 2026

bhuvi27 commented on Jun 16, 2026

vstinner commented on Jun 18, 2026

stestagg commented on Jun 18, 2026

stestagg commented on Jun 18, 2026

vstinner commented on Jun 18, 2026

stestagg commented on Jun 18, 2026

vstinner commented on Jun 24, 2026

vstinner commented on Jun 24, 2026

vstinner commented on Jul 10, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions