Repository navigation
Import _datetime in sub-interpreters in the same time may crash the process #136421
Description
Crash report
What happened?
from concurrent.futures import InterpreterPoolExecutor
executor = InterpreterPoolExecutor()
def f():
import _datetime
print(_datetime)
executor.submit(f)
executor.submit(f)
executor.shutdown()This code may lead to a interpreter crash:
PS C:\Users\xxx\Source\cpython> .\python.bat .\a.py
Running Debug|x64 interpreter...
<module '_datetime' (built-in)> # The first sub-interpreter's output
Assertion failed: !managed_static_type_index_is_set(self), file C:\Users\xxxxx\Source\cpython\Objects\typeobject.c, line 317
Please note that there is a similar issue may crash the interpreter but in the sestructing phase: #136423
My analyze
The _datetime module will call _PyStaticType_InitForExtension to initialize some static types. Look at it's implementation:
Lines 9269 to 9273 in 490eea0
It's using tp_flags Py_TPFLAGS_READY to determine if this type is already initialized. And at last it will call managed_static_type_state_init with the initialized flag.
If the type flag Py_TPFLAGS_READY was not set before, and another sub-interpreter has been initialized, finished, and has set the Py_TPFLAGS_READY flag, this assertion will be triggered:
Lines 316 to 317 in 490eea0
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS, Windows
Output from running 'python -VV' on the command line:
Python 3.15.0a0 (heads/main:ba9c1986305, Jul 8 2025, 22:13:18) [MSC v.1943 64 bit (AMD64)]
Linked PRs
- gh-136421: Fix crash when _datetime is been initialized in multiple sub-interpreters #136422
- gh-136421: Load
_datetimestatic types during interpreter initialization #136583 - gh-136421: Fix crash in _datetime when initialized/finalized concurrently #136620
- [3.14] gh-136421: Load
_datetimestatic types during interpreter initialization (GH-136583) #136943
Activity
aisk commentedon Jul 8, 2025
I have an idea to address this. In the
_datetimemodule's exec phase, we can add a lock to protect these static type initialize process been called in the same time.I don't know if there is a better way to do this, but I think operating on the same (static) object in parallel with multiple reads and writes should be protected by a lock, and it works well on my machine, and I think the static type's initialize process is very fast and will only execute once (per sub-interpreter), so there will be no performance concerns.
The PR is on the way.
aisk commentedon Jul 8, 2025
cc @ericsnowcurrently
ZeroIntensity commentedon Jul 8, 2025
As I said in the PR, I think we need a more general fix that prevents static types from being concurrently initialized. There was a prior attempt at #129829.
neonene commentedon Jul 12, 2025
The immediate workaround in the given script is to import
_datetimemodule first.gh-136421: Load `_datetime` static types during interpreter initializ…
pythongh-136421: Load `_datetime` static types during interpreter ini…
[3.14] gh-136421: Load `_datetime` static types during interpreter in…
kumaraditya303 commentedon Jul 22, 2025
Fixed by #136583
pythongh-136421: Load `_datetime` static types during interpreter ini…
pythongh-136421: Load `_datetime` static types during interpreter ini…
[3.14] pythongh-136421: Load `_datetime` static types during interpre…