Repository navigation
tp_dict slot of static builtin types is NULL in 3.12, without mention in the changelog or an alternative #105227
Description
PyObjC contains some functionality that needs to walk the entire MRO of classes and access the tp_dict slot. As of Python 3.12 beta 1 this code crashes due to the slot being NULL on static builtin types.
The code in PyObjC walks the MRO to implement tp_getattro for the proxy of Objective-C classes to replicate PyObject_GenericGetAttr with some customisations, as wel as in the implementation of an alternative to super (again with customisations in the attribute resolution path). All code paths only need read-only access to the tp_dict of static builtin types, although some code needs to be able to update the tp_dict of classes created by PyObjC.
There is currently no documented alternative for directly accessing tp_dict, and because of this I've changed PyObjC to use the private API _PyType_GetDict.
Some alternatives I've looked into:
-
Use
PyObject_GetAttrString(someType, "__dict__"): This increases the complexity of the PyObjC code bases and requires changes to long stable code:-
Needs to introduce a different code path for accessing the
__dict__slot of non-PyObjC subclasses -
Needs to switch to the abstract Mapping API instead of the PyDict API, which likely has a performance impact
-
Changes to reference counting invariants in the code
-
-
Use
_PyType_GetDicton Python 3.12 or later: This works, but introduces a reliance on a private CPython API that might disappear at any moment (even micro releases), for example by no longer exporting the function from shared libraries.
Proposal: Rename _PyType_GetDict to either PyUnstable_Type_GetDict or PyType_GetDict, while keeping the current interface (e.g. returning a borrowed reference and never raising an exception).
This is a follow-up from #105020
Activity
ronaldoussoren commentedon Jun 2, 2023
@Yhg1s : Would it be acceptable to introduce such a new API in an upcoming 3.12 beta release?
ronaldoussoren commentedon Jun 12, 2023
@Yhg1s, I've added the deferred blocker label because this issue is a seemingly intentional regression from 3.11 without a documented replacement.
I'm currently using a private API and that doesn't feel right.
ericsnowcurrently commentedon Jun 13, 2023
FWIW, I'm in favor of adding
PyType_GetDict(). I've put up a PR: gh-105747.ronaldoussoren commentedon Jun 14, 2023
I agree that its is better to return a new reference instead of a borrowed reference in APIs like this, but that can make life harder for anyone transitioning to the new way of working. In my particular case it does make transitioning harder because I'll have to add refcount updates where I didn't have to do before. That said, this is not a big problem.
pythongh-105227: Add PyType_GetDict() (pythonGH-105747)
gh-105227: Add PyType_GetDict() (GH-105747)
encukou commentedon Jul 10, 2023
@ericsnowcurrently, when you get back, could you review (and maybe improve) the docs?
[3.12] gh-105227: Add PyType_GetDict() (GH-105747) (#106600)
ericsnowcurrently commentedon Jul 17, 2023
The docs seem fine to me, after your changes. My original changes were likely unnecessarily specific. :) Was there something in particular you were concerned about?
ericsnowcurrently commentedon Jul 17, 2023
If you think more should be done, feel free to re-open this.
Fix crash with python 3.12
golang_str: Fix SIGSEGV on import on py3.12