Repository navigation
Runtime errors when launching many shell subprocesses. #93
Description
Here's the code to reproduce those issues:
import uvloop
from asyncio import set_event_loop_policy, create_subprocess_shell, gather, get_event_loop
from asyncio.subprocess import PIPE
async def run_cmd():
length = 1000000
process = await create_subprocess_shell("head -c %s /dev/urandom" % length, stdout=PIPE)
output, err = await process.communicate()
assert len(output) == length
set_event_loop_policy(uvloop.EventLoopPolicy())
loop = get_event_loop()
loop.set_debug(True)
runners = [run_cmd() for i in range(50)]
loop.run_until_complete(gather(*runners))The exact runtime error raised depends on where I run it:
This raises multiple RuntimeError: Cannot pause_reading() when closing:
- uvloop version: 0.8.0
- Python version: 3.6.1
- Platform: linux 4.7.8-1 i686 pae, backport, Debian 8
- Can you reproduce the bug with
PYTHONASYNCIODEBUGin env?: yes
On the same system with python 3.5.3 this runtime error isn't raised but the assertion fails.
This raises multiple exceptions such as RuntimeError: unable to perform operation on <ReadUnixTransport closed=True reading=False 0x7efe0d848d00>; the handler is closed when using /dev/zero instead of /dev/urandom and only needs 2 parallel subprocesses (1 works):
- uvloop version: 0.8.0
- Python version: 3.6.1
- Platform: linux 3.16.0-4-amd64, Debian 8
- Can you reproduce the bug with
PYTHONASYNCIODEBUGin env?: yes
It also raises multiple times RuntimeError: Cannot pause_reading() when closing if run_cmd is launched 200 times instead of 50. Note that this system is on a server with 16G of RAM, while the first one only has 4G (if that makes any difference).
Running the same code with asyncio's default loop works in any case.
Activity
1st1 commentedon May 25, 2017
Interesting. Thanks for providing a code to reproduce this, I'll take a look.
kura commentedon Jul 26, 2017
Might be worth doing a couple of things;
ulimit -a.I'd say while it's unlikely you'll be hitting max processes. if you're running this quite frequently during testing you may easily hit the open files limit, especially if it's causing unclean process shutdown.
FYI, I tested this code on a high spec test rig, found it failed fairly frequently when set to length of
1000000, so I tested it hundreds more times with a random delay between test periods with different length values. All of my tests had zero errors up to and including a length of100000, I tried250000,500000, and750000, all three of these lengths resulting errors with the defaultrange(50)provided in your example.length 10000fails 0length 50000fails 0length 100000fails 0length 250000fails 91/100length 500000fails 89/100length 1000000fails 92/100Make Transport.resume_reading() and pause_reading() idempotent. Add i…
1st1 commentedon Nov 17, 2017
Fixed in master. A simple race between transports and high-level asyncio streams. A new uvloop version with a fix will be released soon.