test__xxsubinterpreters: test_already_running() crash randomly on Python built with TraceRefs: invalid object chain

AMD64 Arch Linux TraceRefs 3.x: https://buildbot.python.org/all/#/builders/484/builds/3721

Assertion error: Objects/object.c:2235: _Py_ForgetReference: Assertion failed: invalid object chain.

$ ./python -m test -j2 test__xxsubinterpreters test__xxsubinterpreters -m test_already_running -v
(...)
== Python build: debug TraceRefs
(...)
== CPU count: 12
(...)
0:00:00 load avg: 1.19 Run tests in parallel using 2 child processes
0:00:00 load avg: 1.19 [1/2] test__xxsubinterpreters passed
test_already_running (test.test__xxsubinterpreters.RunStringTests.test_already_running) ... ok
(...)

0:00:00 load avg: 1.19 [2/2/1] test__xxsubinterpreters crashed (Exit code -6)
test_already_running (test.test__xxsubinterpreters.RunStringTests.test_already_running) ...

Objects/object.c:2235: _Py_ForgetReference: Assertion failed: invalid object chain
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x7f01d0be95a0
object refcount : 0
object type     : 0xa1dc00
object type name: str
object repr     : <refcnt 0 at 0x7f01d0be95a0>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x00007f01df4a1740 (most recent call first):
  <no Python frame>
(...)

Python built with:

git clean -fdx
./configure --with-pydebug --with-trace-refs CFLAGS="-O0"
make -j14

Linked PRs

The object chain is invalid:

(gdb) frame
#13 0x000000000055e731 in _Py_ForgetReference (op=(None, None, None)) at Objects/object.c:2235
2235	        _PyObject_ASSERT_FAILED_MSG(op, "invalid object chain");

(gdb) l
2230	    }
2231	
2232	    if (op == &refchain ||
2233	        op->_ob_prev->_ob_next != op || op->_ob_next->_ob_prev != op)
2234	    {
2235	        _PyObject_ASSERT_FAILED_MSG(op, "invalid object chain");
2236	    }
2237	
2238	#ifdef SLOW_UNREF_CHECK
2239	    PyObject *p;

(gdb) p op->_ob_prev->_ob_next != op
$14 = 1

(gdb) p op->_ob_next->_ob_prev != op
$15 = 1

Similar error in test_interpreters.test_create_many_threaded():

$ ./python -m test test_interpreters -m test_create_many_threaded -v
(...)
test_create_many_threaded (test.test_interpreters.StressTests.test_create_many_threaded) ...

Objects/object.c:2235: _Py_ForgetReference: Assertion failed: invalid object chain
(...)

@ericsnowcurrently

cc @ericsnowcurrently

--with-trace-refs builds use a C global variable to store the linked list of objects. The problem is that the variable is used by all interpreters. The solution is to make that variable per-interpreter.

gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpreters (g…

pythongh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpre…

[3.12] gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpr…

Revert "[3.12] pythongh-107080: Fix Py_TRACE_REFS Crashes Under Isola…

[3.12] gh-107630: Revert "[3.12] gh-107080: Fix Py_TRACE_REFS Crashes…

Unrevert "[3.12] pythongh-107080: Fix Py_TRACE_REFS Crashes Under Iso…

[3.12] gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpr…

@serhiy-storchaka

serhiy-storchaka reopened this Aug 16, 2023

@serhiy-storchaka: Hum, is this issue fixed or not?

I reopened it only because the backport to 3.12 was not merged yet. I linked the backport PR to this issue and thought that merging it will automatically close the issue, but it did not happen.

Navigation Menu

Repository navigation

test__xxsubinterpreters: test_already_running() crash randomly on Python built with TraceRefs: invalid object chain #107080

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

test__xxsubinterpreters: test_already_running() crash randomly on Python built with TraceRefs: invalid object chain #107080

Description

Linked PRs

Activity

vstinner commented on Jul 22, 2023

vstinner commented on Jul 22, 2023

vstinner commented on Jul 22, 2023

ericsnowcurrently commented on Aug 2, 2023

vstinner commented on Sep 14, 2023

serhiy-storchaka commented on Sep 14, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions