Repository navigation
Improve guidelines for GC protocol implementation for heap types #138292
Description
This is a follow-up to #125962 which added some guidelines:
- Use
type->tp_allocinstead ofPyObject_NewandPyObject_GC_New. - Use
type->tp_freeinstead ofPyObject_FreeandPyObject_GC_Del.
Those two recommendations were introduced to facilitate adding the Py_TPFLAGS_HAVE_GC flag to a heap type (those types must (should?) implement the GC protocol, at least according to the docs: https://docs.python.org/3/c-api/gcsupport.html#supporting-cycle-detection).
Now, the docs should indicate that:
type->tp_allocmay callPyObject_GC_Track, depending onPy_TPFLAGS_HAVE_GC. Some users could be surprised by this (like me in gh-116946: fully implement GC protocol forbz2objects #138266).- https://docs.python.org/3.14/c-api/gcsupport.html#supporting-cycle-detection should be updated because it mentions using
PyObject_GC_New, which itself mentions usingtp_allocdirectly. Better to just mentiontp_alloc.
What I am actually worried about is:
Constructors for container types must conform to two rules:
- The memory for the object must be allocated using PyObject_GC_New or PyObject_GC_NewVar.
- Once all the fields which may contain references to other containers are initialized, it must call PyObject_GC_Track().
Now, tp_alloc automatically calls PyObject_GC_Track so users won't be able to pre-initialize fields, so I suggest that we mention this.
Outdated discussion
If people need to first initialize fields, maybe we should recommend constructing them first:
static PyObject *
object_new(PyTypeObject *type)
{
T *self = NULL;
PyObject *f1, *f2, *f3;
f1 = do1();
if (f1 == NULL) { goto error_pre_init; }
f2 = do2();
if (f2 == NULL) { goto error_pre_init; }
f3 = do3();
if (f3 == NULL) { goto error_pre_init; }
self = (T *)type->tp_alloc(type, 0);
if (self == NULL) {
goto error_pre_init;
}
self->f1 = f1;
self->f2 = f2;
self->f3 = f3;
f1 = f2 = f3 = NULL;
if (finalize(self) < 0) {
goto error;
}
return (PyObject *)self;
error_pre_init:
Py_XDECREF(f1);
Py_XDECREF(f2);
Py_XDECREF(f3);
return NULL;
error_post_init:
Py_DECREF(self);
return NULL;
}
Activity
ZeroIntensity commentedon Aug 31, 2025
I don't think this will actually be an issue in practice.
tp_allocwill zero the memory, sotp_clearandtp_traversecan be safely called. If the constructor calls into Python code, the object shouldn't be used anyway, per the note ingc.get_referrers:picnixz commentedon Aug 31, 2025
ZeroIntensity commentedon Aug 31, 2025
picnixz commentedon Sep 1, 2025
In light of #116946 (comment) and #138329 (comment), immutable empty heap types do not need to implement the GC protocol because they will never contribute to a cycle. This may be documented as well.