Repository navigation
[subinterpreters] Multiple interpreters and readline module hook functions. #48452
Description
| BPO | 4202 |
|---|---|
| Nosy | @terryjreedy, @vstinner, @ericsnowcurrently |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2008-10-25.02:52:43.004>
labels = ['expert-subinterpreters', 'type-bug']
title = '[subinterpreters] Multiple interpreters and readline module hook functions.'
updated_at = <Date 2020-05-15.01:28:36.744>
user = 'https://bugs.python.org/grahamd'bugs.python.org fields:
activity = <Date 2020-05-15.01:28:36.744>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Subinterpreters']
creation = <Date 2008-10-25.02:52:43.004>
creator = 'grahamd'
dependencies = []
files = ['27727']
hgrepos = []
issue_num = 4202
keywords = ['patch']
message_count = 5.0
messages = ['75201', '107981', '173809', '173839', '368905']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'vstinner', 'grahamd', 'Winston451', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue4202'
versions = ['Python 3.5', 'Python 3.6']
Activity
grahamd commentedon Oct 25, 2008
Because the readline module uses PyGILState_Ensure() to facilitate triggering
callbacks into Python code, this would make the ability to use the hook
functions incompatible with use in sub interpreters.
If this is the case, then that readline module cannot be used in sub
interpreters should be documented if not already.
Better still, attempts to register hooks from sub interpreters should result in
an exception. Further, when in use, in sub interpreter, callback hooks should
also not be called if defined, because if defined they would be the hooks from
the main interpreter since variables holding the hooks are static C variables
and shared across all interpreters.
This issue derived from reading of code only and not tested in real program.
terryjreedy commentedon Jun 17, 2010
The be an issue, tests with currents interpreters are needed. You may be right, but it may be that no one ever uses readline with subinterpreters.
Winston451 commentedon Oct 25, 2012
Hi,
I'm currently developping an application which runs sub interpreters in threads (one sub interpreter per thread). When I try to use readline completion in these sub interpreters it fails because it's not possible to import the "keyword" module (PyEval_GetRestricted() returns true). I think it is due to the fact that readline's on_completion() use PyGILState_Ensure(). It should be possible to use PyEval_RestoreThread with _PyOS_ReadlineTState instead.
Winston451 commentedon Oct 26, 2012
I attached a patch that solve the problem
[-]Multiple interpreters and readline module hook functions.[/-][+][subinterpreters] Multiple interpreters and readline module hook functions.[/+]vstinner commentedon May 15, 2020
See also bpo-15751: [subinterpreters] Make the PyGILState API compatible with subinterpreters.
ericsnowcurrently commentedon Jul 8, 2024
In the end, we kept the readline module single-phase init, which means it cannot be used in "isolated" subinterpreters (own GIL).
Furthermore, @tonybaloney landed gh-112313 to resolve some of the complexity of the callbacks. I landed gh-121060 (3.13+) to force extensions to always be loaded in the main interpreter first, which resolves some of the other possible complexities. Overall, I believe this issue is sorted out.
If we later want to work toward making the readline module work in multiple interpreters then I recommend we open a new issue.