Repository navigation
Crash in repr() for lists containing NULLs #146056
Description
Activity
- added a commit that references this issue
on Mar 17, 2026 Missing Py_NewRef causes double-free in treebuilder_handle_end
It's not a double free, but the TreeBuilder stack list which contains NULL items.
repr(stack)crashes sincelist_repr()makes the assumptions that all list items are notNULL.Also,
treebuilder_handle_end()has a surprising code to handle references, but the current "works" (by luck):item = self->last; self->last = Py_NewRef(self->this); Py_XSETREF(self->last_for_tail, self->last); self->index--; self->this = Py_NewRef(PyList_GET_ITEM(self->stack, self->index)); Py_DECREF(item);
last_for_tailassignment consumes one reference tothis, but thenthisis set without decrementing its reference count (no Py_SETREF, no Py_DECREF): solast_for_tailref count is correct at the end.While the code works (by luck), I prefer to change it to make reference counting more explicit.
I wrote PR gh-146062 to fix TreeBuilder stack and fix
treebuilder_handle_end()reference counting.Other way to fix this is to make the list's repr safe for NULLs. This is not the only place were we preallocate a non-empty list, and in the GIL-less build we can expect more random crashes when inspecting
gc.get_referrers(),gc.get_referents()orgc.get_objects()or playing with debugger.I would prefer to leave
repr(tuple)andrepr(list)unchanged if possible.This is not the only place were we preallocate a non-empty list, and in the GIL-less build we can expect more random crashes when inspecting gc.get_referrers(), gc.get_referents() or gc.get_objects() or playing with debugger.
We should fix these issues on a case by case basis.
Reacted by Arseniy TerekhinThis was a regression in
list.__repr__()introduced in 3.13.Oh,
PyObject_Repr(NULL)already returns the string"<NULL>", I didn't know that.This was a regression in list.repr() introduced in 3.13.
Oh, I added this regression in commit 3de0f55 (fix for another bug). Well, there was no test for
repr(list)withNULL:-)Reacted by Arseniy TerekhinThere were two breaking changes -- adding incref and using the PyUnicodeWriter C API. The latter broke also the repr of uninitialized tuples and perhaps other objects. So the simplest way is to fix
PyUnicodeWriter_WriteRepr(), otherwise we would need to add a workaround for each use ofPyUnicodeWriter_WriteRepr()(and leave unexploded bombs in the user code).- changed the title
[-]Double-free in `xml.etree.ElementTree.TreeBuilder`[/-][+]Crash in repr() for lists containing NULLs[/+]on Mar 20, 2026 - added 5 commits that reference this issue
on Mar 22, 2026 The reproducer does no longer crash:
[[None, <Element 'a' at 0x7fb7d75f9310>, <Element 'a' at 0x7fb7d73cf5f0>, <Element 'a' at 0x7fb7d73cf6b0>, <Element 'a' at 0x7fb7d73cf770>, <Element 'a' at 0x7fb7d73cf830>, <Element 'a' at 0x7fb7d73cf8f0>, <Element 'a' at 0x7fb7d73cf9b0>, <Element 'a' at 0x7fb7d73cfa70>, <Element 'a' at 0x7fb7d73cfb30>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>], <Element 'a' at 0x7fb7d75f9310>]Instead, it displays the stack list which contains
<NULL>items.Thanks @devdanzin for your bug report and thanks @serhiy-storchaka for the generic fix (backported to 3.13 and 3.14 branches).
- added a commit that references this issue
on Mar 23, 2026
Crash report
What happened?
It's possible to segfault the interpreter from a double-free in
xml.etree.ElementTree.TreeBuilder. Please let me know if the automated diagnosis is incorrect and whether you prefer me not to post it in new issues.Automated diagnosis:
Missing Py_NewRef causes double-free in treebuilder_handle_end (line 2851):
Py_XSETREF(self->last_for_tail, self->last)stores intolast_for_tailwithout taking a new reference. Both fields point to the same object with only one reference.treebuilder_gc_cleardecrements twice — a double-free triggered on every XML end tag. Lines 2882 and 2922 demonstrate the correct pattern withPy_NewRef.Fix:
Py_XSETREF(self->last_for_tail, Py_NewRef(self->last));MRE:
Backtrace:
Claude explanation of the MRE and crash:
builder.end("a")10 times, each time executing line 2851:Py_XSETREF(self->last_for_tail, self->last)— missingPy_NewRef. Each end tag over-decrements the element it touches.builder.close(),root[0](the innermost<a>) has had its refcount corrupted — it's lower than it should be.gc.get_referrers(root[0])returns a list containing objects that reference that element. One of those referrers is theTreeBuilder's internal stack list, which itself contains elements with corrupted refcounts.print()callslist_repron that referrers list, which callsPy_NewRefon each item. When it hits an element whose refcount was decremented to 0 (already freed),opisNULLor points to freed memory → segfault atPy_INCREF(op=0x0).The backtrace confirms it:
list_repr_implatlistobject.c:604tries toPy_NewRefaNULLpointer — an object that was prematurely freed due to the cumulative over-decrements from the 10end()calls.Found using cpython-review-toolkit.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.15.0a7+ (heads/main:99e2c5eccd2, Mar 17 2026, 08:26:50) [Clang 21.1.2 (2ubuntu6)]
Linked PRs