Repository navigation
Script using ProactorEventLoop does not exit cleanly when subprocesses are spawned. #81562
Closed as not planned
Description
| BPO | 37381 |
|---|---|
| Nosy | @pfmoore, @tjguk, @asvetlov, @zware, @1st1, @zooba, @cr1901 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2019-06-23.22:35:46.548>
labels = ['type-bug', '3.7', 'OS-windows', 'expert-asyncio']
title = 'Script using ProactorEventLoop does not exit cleanly when subprocesses are spawned.'
updated_at = <Date 2019-06-24.01:07:24.313>
user = 'https://github.com/cr1901'bugs.python.org fields:
activity = <Date 2019-06-24.01:07:24.313>
actor = 'cr1901'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Windows', 'asyncio']
creation = <Date 2019-06-23.22:35:46.548>
creator = 'cr1901'
dependencies = []
files = []
hgrepos = []
issue_num = 37381
keywords = []
message_count = 2.0
messages = ['346345', '346346']
nosy_count = 7.0
nosy_names = ['paul.moore', 'tim.golden', 'asvetlov', 'zach.ware', 'yselivanov', 'steve.dower', 'cr1901']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue37381'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']
Activity
cr1901 commentedon Jun 23, 2019
On Windows 7, when using ProactorEventLoop, when asyncio.subprocess.terminate is called on the last subprocess when multiple subprocesses are spawned*, the last subprocess will never exit, and the Python interpreter will never exit.
Spawning an instance of "TASKKILL /F" to kill the subprocess, and then awaiting asyncio.subprocess.wait is a workaround. Since both "TASKKILL /F" and asyncio.subprocess.terminate ultimately call TerminateProcess, I would expect the same behavior using both (without the need to wait for the subprocess to actually die).
I have confirmed this behavior on Python 3.5, 3.6, and 3.7 on both MSVC and MinGW Python (although I most frequently use the latter).
Zip file is too large, so I have a GH repo (ready to run) demonstrating this issue: https://github.com/cr1901/proactor
An example run looks like the following:$ python3 proactor.py
Infinite Loop, Iteration 0
Finite Loop, Iteration 0
Infinite Loop, Iteration 1
Finite Loop, Iteration 1
Infinite Loop, Iteration 2
Finite Loop, Iteration 2
Finite Loop, Iteration 3
Infinite Loop, Iteration 3
Infinite Loop, Iteration 4
Task poller finished.
The interpreter hangs after the "Task poller finished." line, which makes me believe that the Python interpreter is trying and failing to cleanup resources for some reason (IOCP never completes?).
*Each subprocess is associated with its own coroutine to handle standard output.
cr1901 commentedon Jun 23, 2019
Update: It turns out multiple spawned subprocesses are NOT required. If I replace "(short_fut, _) = await spawn()" with "short_fut = asyncio.ensure_future(asyncio.sleep(4))", the hang still occurs.
[-]Script using ProactorEventLoop does not exit cleanly on Windows with multiple subprocesses[/-][+]Script using ProactorEventLoop does not exit cleanly when subprocesses are spawned.[/+]arhadthedev commentedon Jun 30, 2022
If this issue is fixed by gh-92841, it should be closed.
kumaraditya303 commentedon Oct 7, 2022
@cr1901 Can you check if this is still an issue on main branch and 3.11? It would be better if you can provide a reproducer without involving rust.
cr1901 commentedon Oct 7, 2022
@kumaraditya303 I cannot duplicate this on 3.10.7; I'm not in a position anymore to compile and test the main branch of Python (or 3.11) at present.
There is still a minor bug where if I run the script before both
echoprocesses have terminated, the script runs a single iteration and exits with code120. I can't tell if it's related to this bug or a separate one.I don't remember the rationale here. I think I was having trouble spawning a python process taking a script as input. I'll see what I can do to make the reproducer Python-only.
kumaraditya303 commentedon Oct 7, 2022
asynciosubprocess is significantly improved in 3.11 and 3.12 (main). A lot of bugs have been fixed so it is likely the bug is already fixed. I look forward to a Python only reproducer.kumaraditya303 commentedon Oct 26, 2022
I tried your rust reproducer and the bug is fixed in main, 3.11 and 3.10 it doesn't hangs so closing as it is fixed.