Repository navigation
Segmentation fault during interpreter finalization in PyTraceMalloc_Untrack #129185
Description
Crash report
What happened?
_PyTraceMalloc_Fini is called before _PyImport_Fini in _Py_Finalize. This can lead to crashes when using tracemalloc if calls to PyTraceMalloc_Untrack are performed during finalization of module (e.g in the destructor of some global variable).
Commit 6b47499 moved the call to TABLES_LOCK before the check tracemalloc_config.tracing so now the issue whether or not tracemalloc is started.
The following extension module reproduces the issue:
#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include <numpy/arrayobject.h>
static PyObject* arr = NULL;
static struct PyModuleDef repromodule = {
PyModuleDef_HEAD_INIT,
"repro",
NULL,
-1,
NULL
};
PyMODINIT_FUNC
PyInit_repro(void)
{
PyObject* m;
m = PyModule_Create(&repromodule);
if (!m)
return NULL;
import_array();
arr = PyArray_SimpleNew(1, (npy_intp[]){0}, NPY_DOUBLE);
if (!arr || PyModule_AddObject(m, "arr", arr) < 0) {
Py_XDECREF(arr);
Py_DECREF(m);
return NULL;
}
return m;
}$ python -c 'import repro'It declares a global NumPy array and array_dealloc calls PyTraceMalloc_Untrack, creating a segfault.
For context, I initially ran into the issue in a pybind11 extension declaring functions with numpy array as defaults arguments.
CPython versions tested on:
CPython main branch, 3.13
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.13.1+ (heads/3.13:a3797492179, Jan 22 2025, 11:46:15) [GCC 11.4.0]
Linked PRs
- gh-129185: Fix crash during interpreter finalization in PyTraceMalloc_Untrack #129188
- gh-129185: Fix PyTraceMalloc_Untrack() at Python exit #129191
- [3.13] gh-129185: Fix PyTraceMalloc_Untrack() at Python exit (#129191) #129217
- gh-129185: Remove internal TRACE_RAW_MALLOC macro #129218
- [3.12] gh-129185: Fix PyTraceMalloc_Untrack() at Python exit (#129191) (#129217) #129221
- gh-129185: Use PyMutex in tracemalloc #129246
- gh-129185: Simplify PyTraceMalloc_Track() #129256
Activity
picnixz commentedon Jan 22, 2025
Btw,
PyModule_AddObjectsteals a reference, so you might want to increfarrbefore since you're storing it in a global variable but I don't know whether this would solve the issue. Now, can you actually produce a reproducer that does not depend onnumpyso that we can possibly add a regression test?cc @vstinner
pythongh-129185: Fix PyTraceMalloc_Untrack() at Python exit
pythongh-129185: Fix PyTraceMalloc_Untrack() at Python exit
vstinner commentedon Jan 22, 2025
I wrote a different fix with a test: PR gh-129191.
gh-129185: Fix PyTraceMalloc_Untrack() at Python exit (#129191)
pythongh-129185: Fix PyTraceMalloc_Untrack() at Python exit (python#1…
pythongh-129185: Remove internal TRACE_RAW_MALLOC macro
gh-129185: Remove internal TRACE_RAW_MALLOC macro (#129218)
[3.13] gh-129185: Fix PyTraceMalloc_Untrack() at Python exit (#129191) (
vstinner commentedon Jan 23, 2025
Thanks @rostan-t for the bug report and for the initial fix! It's now fixed in 3.13 and main branches, and the 3.12 fix is incoming.
[3.12] gh-129185: Fix PyTraceMalloc_Untrack() at Python exit (#129191) (
pythongh-129185: Use PyMutex in tracemalloc
pythongh-129185: Use PyMutex in tracemalloc
gh-129185: Use PyMutex in tracemalloc (#129246)
pythongh-129185: Simplify PyTraceMalloc_Track()
pythongh-129185: Simplify PyTraceMalloc_Track()
gh-129185: Simplify PyTraceMalloc_Track() (#129256)