Repository navigation
MainThread association logic is fragile #75698
Description
| BPO | 31517 |
|---|---|
| Nosy | @tim-one, @ncoghlan, @pitrou, @vstinner, @fabioz, @asvetlov, @ericsnowcurrently, @aldwinaldwin, @int19h |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2017-09-19.13:00:11.242>
labels = ['expert-subinterpreters', 'type-feature', '3.8', '3.9', 'docs']
title = 'MainThread association logic is fragile'
updated_at = <Date 2020-06-03.16:42:33.468>
user = 'https://github.com/pitrou'bugs.python.org fields:
activity = <Date 2020-06-03.16:42:33.468>
actor = 'vstinner'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Subinterpreters']
creation = <Date 2017-09-19.13:00:11.242>
creator = 'pitrou'
dependencies = []
files = ['47153']
hgrepos = []
issue_num = 31517
keywords = []
message_count = 11.0
messages = ['302521', '302523', '302583', '302589', '302601', '302602', '302603', '302606', '347264', '347266', '358744']
nosy_count = 10.0
nosy_names = ['tim.peters', 'ncoghlan', 'pitrou', 'vstinner', 'fabioz', 'asvetlov', 'docs@python', 'eric.snow', 'aldwinaldwin', 'int19h']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue31517'
versions = ['Python 3.8', 'Python 3.9']
Activity
pitrou commentedon Sep 19, 2017
The threading main_thread() instance is associated to the thread that first imports the threading module. In usual circumstances, this will indeed be the interpreter's main thread. However, it is not always the case. See attached reproducer.
pitrou commentedon Sep 19, 2017
Fixing this will require thinking out what the "main thread" of an interpreter really is. Is it the thread in which PyInterpreterState_New() is called?
(note this is not the same thing as the "main thread" in the signal module, which is process-global)
tim-one commentedon Sep 20, 2017
Is there a problem here? I haven't heard of anyone even wondering about this before. threading.py worries about Threads created by threading.py, plus the magical (from its point of view) thread that first imports threading.py. Users mix in
_threadthreads, or raw C threads from extension modules, essentially at their own risk. Which risks are minimal, but can have visible consequences.I don't view that as being a real problem. It might help if, e.g., a Wiki somewhere spelled out the consequences under different Python implementations (while I don't know for sure, I _expect_ the current docs just say "in normal conditions, the main thread is the thread from which the Python interpreter was started" because it doesn't want to over-specify the behavior in an area nobody really cares about).
ncoghlan commentedon Sep 20, 2017
One place where this came up recently is in working out precisely how a Python-level subinterpreter API will interact with the threading API: https://mail.python.org/pipermail/python-dev/2017-September/149566.html
That said, I do agree with Tim that the status quo isn't broken per se: we renamed
threadto_threadin Python 3 for a reason, and that reason is that you really need to know how Python's threading internals work to do it safely.However, I do think we can treat this as a documentation enhancement request, where the
_threadmodule docs could point out some of the requirements to ensure that low-level thread manipulation plays nice with the threading module.pitrou commentedon Sep 20, 2017
This came while working on the PEP-556 implementation.
pitrou commentedon Sep 20, 2017
This is not caused by
_threadspecifically, but any background thread created by C code that might invoke Python code. The reproducer just uses the_threadmodule out of convenience.pitrou commentedon Sep 20, 2017
I'm not sure this is an issue worth fixing, but I don't think it's a good idea to document such quirky semantics.
ncoghlan commentedon Sep 20, 2017
We had the quirks of import related threading deadlocks documented for a long time, not as a promise, but as a debugging aid (and a recommendation for "don't do that").
I'd see this as being similar: we'd document that if you're using the _thread module, or otherwise allowing operating system threads not managed by the threading module to call in and run arbitrary Python code, then you should run "import threading" early in the actual main thread to make sure it gets associated correctly.
vstinner commentedon Jul 4, 2019
Python internals already know who is the "main" thread: _PyRuntime.main_thread. It's maintained up to date, even after a fork, PyOS_AfterFork_Child() calls _PyRuntimeState_ReInitThreads() which does:
I already added _thread._is_main_interpreter() to deny spawning daemon threads in subinterpreters: bpo-37266.
We can add _thread._is_main_thread() which can reuse Modules/signalmodule.c code:
For example, this function is used by signal.signal:
vstinner commentedon Jul 4, 2019
bpo-37416 has been marked as a duplicate of this issue. It contains snippet.py to reproduce a bug.
ericsnowcurrently commentedon Dec 20, 2019
probably a duplicate: issue bpo-39042 "Use the runtime's main thread ID in the threading module."