Repository navigation
[subinterpreters] Per-interpreter singletons (None, True, False, etc.) #83692
Description
Activity
The long-term goal of the PEP-554 is to run two Python interpreters in parallel. To achieve this goal, no object must be shared between two interpreters. See for example my article "Pass the Python thread state explicitly" which gives a longer rationale:
https://vstinner.github.io/cpython-pass-tstate.htmlIn bpo-38858, I modified Objects/longobject.c to have per-interpreter small integer singletons: commit 630c8df.
This issue is about other singletons like None or Py_True which are currently shared between two interpreters.
I propose to add new functions. Example for None:
- Py_GetNone(): return a *borrowed* reference to the None singleton (similar to existing Py_None macro)
- Py_GetNoneRef(): return a *strong* reference to the None singleton (similar to "Py_INCREF(Py_None); return Py_None;" and Py_RETURN_NONE macro)
And add PyInterpreterState.none field: strong reference to the per-interpreter None object.
We should do that for each singletons:
- None (Py_None)
- True (Py_True)
- False (Py_False)
- Ellipsis (Py_Ellipsis)
GIL issue
=========Py_GetNone() would look like:
PyObject* Py_GetNone(void) { return _PyThreadState_GET()->interp->none; }
Problem: _PyThreadState_GET() returns NULL if the caller function doesn't hold the GIL.
Using the Python C API when the GIL is not held is a violation of the API: it is not supported. But it worked previously.
One solution is to fail with an assertion error (abort the process) in debug mode, and let Python crash in release mode.
Another option is to only fail with an assertion error in debug mode in Python 3.9. In Python 3.9, Py_GetNone() would use PyGILState_GetThisThreadState() function which works even when the GIL is released. In Python 3.10, we would switch to _PyThreadState_GET() and so crash in release mode.
One concrete example of such issue can be found in the multiprocessing C code, in semlock_acquire():
Py_BEGIN_ALLOW_THREADS if (timeout_obj == Py_None) { res = sem_wait(self->handle); } else { res = sem_timedwait(self->handle, &deadline); } Py_END_ALLOW_THREADS
Py_None is accessed when the GIL is released.
Would it not suffice to just make the singletons "immortal"?
Without affecting the hotpaths that are Py_INCREF and Py_DECREF, changing _Py_Dealloc to test for objects with a "special" destructor could be used:
destructor dealloc = Py_TYPE(op)->tp_dealloc; if (dealloc == _Py_SingletonSentinel) { /* reset refcnt so as to not return here too often */ op->ob_refcnt = PY_SSIZE_T_MAX; } else { (*dealloc)(op); }
Even in the presence of multiple mutating threads, the object cannot be destroyed. Worst case, they all call _Py_Dealloc.
Would it not suffice to just make the singletons "immortal"?
The problem is to make Py_INCREF/Py_DECREF efficient. Last time someone tried to use an atomic variable for ob_refcnt, it was 20% slower if I recall correctly. If many threads start to update such atomic variable, the CPU cacheline of common singletons like None, True and False can quickly become a performance bottleneck.
On the other side, if each interpreter has its own objects, there is no need to protect ob_refcnt, the interpreter lock protects it.
jeremykloth commented
on Feb 1, 2020 jeremyklothmannequinMannequinThe problem is to make Py_INCREF/Py_DECREF efficient.
That is exactly why I didn't propose a change to them. The singletons
still are refcounted as usual, just that their ob_refcnt is ignored.
If they somehow reach 0, they just "resurrect" themselves and ignore
the regular collection behavior. In the presence of multiple
DECREF'ers, the ob_refcnt field is garbage, but that is OK as it is
effectively ignored. Practicality vs Purity and all that.Last time someone tried to use an atomic variable for ob_refcnt, it was 20% slower if I recall correctly. If many threads start to update such atomic variable, the CPU cacheline of common singletons like None, True and False can quickly become a performance bottleneck.
Exactly so, hence why I chose the simple solution of effectively
ignoring ob_refcnt.On the other side, if each interpreter has its own objects, there is no need to protect ob_refcnt, the interpreter lock protects it.
My solution also does not need any protection around ob_refcnt.
I vaguely recall discussions about immortal Python objects.
(*) Instagram gc.freeze()
- https://docs.python.org/dev/library/gc.html#gc.freeze
- https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172
(*) Python immortal strings
- PyUnicode_InternImmortal()
- SSTATE_INTERNED_IMMORTAL
- They are only destroyed if Python is compiled with Valgrind or Purify support: unicode_release_interned() function
(*) COUNT_ALLOCS
- When Python is built with COUNT_ALLOCS macro defined, types are immortal
- Many tests have to be skipped if COUNT_ALLOCS is used
- I plan to remove COUNT_ALLOCS feature in bpo-39489 :-)
(*) Static types
Recently, Petr Viktorin proposed immortal singletons in my latest "Pass the Python thread state to internal C functions" thread on python-dev list:
https://mail.python.org/archives/list/python-dev@python.org/message/RAVSH7HYHTROXSTUR3677WGTCTEO6FYF/In 2004, Jewett, Jim J proposed:
"What if a few common (constant, singleton) objects (such as None, -1, 0, 1) were declared immortal at compile-time?"
Is the sub-interpreter PEP approved? If not, I had thought the plan was to only implement PRs that made clean-ups that would have been necessary anyway.
Random idea (not carefully thought-out): Would it be simpler to have these objects just ignore their refcount by having dealloc() be a null operation or having it set the refcount back to a positive number). That would let sub-interpreters share the objects without worrying about race-conditions on incref/decref operations. To make this work, the objects can register themselves as permanent, shared, objects; then, during shutdown, we could explicitly call a hard dealloc on those objects.
16 remaining items
Those numbers are for code without immortal objects.
They don't apply in this case, as the branch misprediction rate would rise.JunyiXie commented
on Mar 10, 2021 JunyiXiemannequinMannequinWhich API should be used in C extensions to be "subinterpreter-safe"? ?> Currently, Py_None is a singleton shared by multiple interpreters. > > > Should suddenly all C extensions use a new Py_GetNone() function which > returns the per-interpreter singleton? If yes, that's basically what my > PR 18301 does:
#define Py_None Py_GetNone()
after read you [WIP] bpo-39511: Add Py_GetNone() and Py_GetNoneRef() functions bpo-18301.
Actually, interp->none shared _Py_NoneStruct variable.
when two interperter modify interp->none refcount,will modify _Py_NoneStruct variable.
the CPU cacheline of common singletons like None, True and False can quickly become a performance bottleneck.
even if add Py_INCREF(none);.
In the scenario of parallel interpreter, will also have thread safety issues.PyStatus
_Py_InitSingletons(PyThreadState *tstate)
{
PyObject *none = &_Py_NoneStruct;
Py_INCREF(none);
tstate->interp->none = none;
return _PyStatus_OK();
}Actually, interp->none shared _Py_NoneStruct variable.
My PR 18301 is a draft to check if we can solve the issue without breaking the C API compatibility. You're right that it doesn't solve the issue, it only checks the C API issue. IMO the PR 18301 proves that the "#define Py_None Py_GetNone()" trick works.
--
By the way, when I worked on a tagged pointer experiment:
vstinner#6I had to introduce Py_IS_NONE(op) function, since it was no longer possible to compare directly "op == Py_None".
static inline int Py_IS_NONE(PyObject *op) { return (op == &_Py_NoneStruct || op == _Py_TAGPTR_NONE); }
But this is not needed to solve this issue.
Shouldn't this wait to see if the subinterpreters PEP is approved? Because if it isn't, then no chance should be made. We shouldn't change something this fundamental without good cause.
Raymond Hettinger: "Shouldn't this wait to see if the subinterpreters PEP is approved? Because if it isn't, then no chance should be made. We shouldn't change something this fundamental without good cause."
I agree that we reached a point where a PEP is needed before pushing further "controversial" changes related to subinterpreters and bpo-1635741 (especially converting static types to heap types (bpo-40077).
I plan to write multiple PEPs:
KubaO commented
on Jan 24, 2022 KubaOmannequinMannequinI'm looking very much forward to isolated subinterpreters and thus the per-subinterpreter GIL, as I've been keeping a private exploratory project where I had to make them work.
Here are my thoughts:
-
Any sort of reference count on heavily used objects cannot be shared between the threads, even if its value is otherwise ignored. I.e., a write-only shared refcount is already a no-no. The mere fact that it's being modified from different threads is a performance bottleneck, as the cacheline that holds the refcount has to be shuttled between cores. That's a bad thing and the penalty only becomes worse as the time marches on.
-
For platforms where the C language supports thread-local storage (TLS) at the declaration level, it's trivial to have "global static" immortal objects become thread-local static. This could be perhaps an escape to keep old code working to an extent, as opposed to immediately breaking. On such platforms, the
PyGet_FooAPI can be on equal footing with the legacyPy_Foostatics, i.e. both would do the same thing. That's how I've done it in my experiment. The obvious problem is that on platforms without compiler support for TLS,Py_Foowould be unavailable, and that's probably a no-go for an API that wouldn't be deprecated. For portability, everyone should be usingPyGet_Foo, iff platforms without language-level TLS support are of interest (are they? what would they be?)
-
On such platforms, the
PyGet_FooAPI can be on equal footing with the legacyPy_Foostatics, i.e. both would do the same thing. That's how I've done it in my experiment. The obvious problem is that on platforms without compiler support for TLS,Py_Foowould be unavailable, and that's probably a no-go for an API that wouldn't be deprecated.My #62501 PR uses "#define Py_None Py_GetNone()" which is backward compatible in terms of API.
Py_GetNone() can have various implementations, it doesn't matter at the API level.
It seems like most core devs prefer https://peps.python.org/pep-0683/ (even if it's still a draft). I close this issue.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: