asyncio: nested event loop

@jcea

BPO	22239
Nosy	@jcea, @njsmith, @jab, @pmp-p, @vadmium, @1st1, @crusaderky, @douglas-raillard-arm, @davidbrochart, @kumaraditya303
Files	nested.py: Decorator allowing a coroutine to run in a nested event loop nested.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-08-25.18:04:32.900>
created_at = <Date 2014-08-20.18:03:50.601>
labels = ['type-feature', 'expert-asyncio']
title = 'asyncio: nested event loop'
updated_at = <Date 2021-12-11.07:40:23.171>
user = 'https://bugs.python.org/djarb'

bugs.python.org fields:

activity = <Date 2021-12-11.07:40:23.171>
actor = 'kumaraditya'
assignee = 'none'
closed = True
closed_date = <Date 2014-08-25.18:04:32.900>
closer = 'gvanrossum'
components = ['asyncio']
creation = <Date 2014-08-20.18:03:50.601>
creator = 'djarb'
dependencies = []
files = ['36422', '36498']
hgrepos = []
issue_num = 22239
keywords = ['patch']
message_count = 23.0
messages = ['225578', '225883', '226037', '298388', '373145', '373177', '373183', '373186', '373192', '373240', '373241', '373244', '373245', '373247', '373249', '373270', '373273', '373280', '373307', '373311', '389487', '398257', '399920']
nosy_count = 14.0
nosy_names = ['jcea', 'djarb', 'zzzeek', 'njs', 'jab', 'pmpp', 'martin.panter', 'yselivanov', 'Rokas K. (rku)', 'crusaderky', 'douglas-raillard-arm', 'davidbrochart', 'kumaraditya', 'rob.moore']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue22239'
versions = []

Linked PRs

gh-66435: Allow nested event loops #93338

It's occasionally necessary to invoke the asyncio event loop from code that was itself invoked within (although usually not directly by) the event loop.

For example, imagine you are writing a class that serves as a local proxy for a remote data structure. You can not make the __contains__ method of that class into a coroutine, because Python automatically converts the return value into a boolean. However, __contains__ must invoke coroutines in order to communicate over the network, and it must be invokable from within a coroutine to be at all useful.

If the event loop _run_once method were reentrant, addressing this problem would be simple. That primitive could be used to create a loop_until_complete function, which could be applied to the io tasks that __contains__ needs to invoke

So, making _run_once reentrant is one way of addressing this request.

Alternately, I've attached a decorator that sets aside some of the state of _run_once, runs a couroutine to completion in a nested event loop, restores the saved state, and returns the coroutine's result. This is merely a proof of concept, but it does work, at least in my experiments.

While I understand your problem, I really do not want to enable recursive event loops. While they are popular in some event systems (IIRC libevent relies heavily on the concept), I have heard some strong objections from other parts, and I am trying to keep the basic event loop functionality limited to encourage interoperability with other even loop systems (e.g. Tornado, Twisted).

In my own experience, the very programming technique that you are proposing has caused some very hard to debug problems that appeared as very infrequent and hard to predict stack overflows.

I understand this will make your code slightly less elegant in some cases, but I think in the end it is for the best if you are required to define an explicit method (declared to be a coroutine) for membership testing of a remote object. The explicit "yield from" will help the readers of your code understand that global state may change (due to other callbacks running while you are blocked), and potentially help a static analyzer find bugs in your code before they take down your production systems.

All right.

However, for anyone who's interested, here is a patch that enables nested event loops in asyncio, and the accompanying unit tests

I understand rationale for rejection of this issue but i beg to reconsider.

Unlike in traditional coroutines (windows fibers / setjmp|longjmp with stack switching) we can not yield from any point of execution. There must be full async-await chain preserved. This basically divides code into two islands - async and non-async. And there seems to be no way to schedule async call from non-async code and get a response. While suggestion to make custom async def contains() call is a valid one we can not always do that. Consider the case when we have to do some networking calls in a function that is invoked by non-async library. Naturally it would be simple non-awaited call from which we can not call a coroutine and get a response. And since it is a library calling into our code we can not easily change it. It might even be completely unsuitable change for library in question.

I see two solutions to this problem (if i am missing something please point it out):

Reentrant loops as suggested in this issue.
Allow awaited calls from non-coroutines provided execution is invoked from a coroutine somewhere up the call stack.

First one is certainly easier to implement.

hey there,

I seem to have two cents to offer so here it is. An obscure issue in the Python bug tracker is probably not the right place for this so consider this as an early draft of something that maybe I'll talk about more elsewhere.

This basically divides code into two islands - async and non-async

yes, this is the problem, and at the bottom of this apparently somewhat ranty comment is a solution, and the good news is that it does not require Python or asyncio be modified. My concern is kind of around how it is that everyone has been OK with the current state of affairs for so long, why it is that "asyncio is fundamentally incompatible with library X" is considered to be acceptable, and also how easy it was to find a workaround, this is not something I would have expected to come up with. Kind of like you don't expect to invent Velcro or windshield wipers.

asyncio's approach is what those of us in the library/framework community call "explicit async", you have to mark functions that will be doing IO and the points at which IO occurs must also be marked. Long ago it was via callback functions, then asyncio turned it into decorators and yields, and finally PEP-492 turned it into async/await, and it is very nicely done. It is of course a feature of asyncio that writing out async/await means your code can in theory be clearer as to where IO occurs and all that, and while I don't totally buy that myself, I'm of course in favor of that style of coding being available, it definitely has its own kind of self-satisfaction built in when you do it. That's all great.

But as those of us in the library/framework community also know, asyncio's approach essentially means, libraries like Flask, Django, my own SQLAlchemy, etc. are all automatically "non-workable" with the asyncio approach; while these libraries can certainly have asyncio endpoints added to them, the task as designed is not that simple, since to go from an asyncio endpoint all the way through library code that doesn't care about async and then down into a networking library that again has asyncio endpoints, the publishing of "async" and the "await" or yield approach must be wired all the way through every function and method. This is all despite that when you're not at the endpoints, the points at which IO occurs is fully predictable such that libraries like gevent don't need you to write it. So we are told that libraries have to have full end-to-end rewrites of all their code to work this way, or otherwise maintain two codebases, or something like that.

The side effect of this is that a whole bunch of library and framework authors now get to create all new libraries and frameworks, which do exactly the same thing as all the existing libraries and frameworks, except they sprinkle the "async/await" keywords throughout middle tiers as required. Vague claims of "framework X is faster because it's async" appear, impossible to confirm as it is unknown how much of their performance gains come from the "async" aspect and how much of it is that they happened to rewrite a new framework from scratch in a completely different way (hint: it's the latter).

Or in other cases, as if to make it obvious how much the "async/await" keywords come down to being more or less boilerplate for the "middle" parts of libraries, the urllib3 project wrote the "unasync" project [1] so that they can simply maintain two separate codebases, one that has "async/await" and the other which just search-and-replaced them out.

SQLAlchemy has not been "replaced" by this trend as asyncio database libraries have not really taken off in Python, and there are very few actual async drivers. Some folks have written SQLAlchemy-async libraries that use SQLAlchemy's expression system while they have done the tedious, redundant and impossible-to-maintain work of replicating enough of SQLAlchemy's execution internals such that a modest "sqlalchemy-like" experience with asyncio can be reproduced. But these libraries are closed out from all of the fixes and improvements that occur to SQLAlchemy itself, as well as that these systems likely target a smaller subset of SQLAlchemy's behaviors and features in any case. They certainly can't get the ORM working as the ORM runs lots of SQL executions internally, all of which would have to propagate their "asyncness" outwards throughout hundreds of functions.

The asyncpg project, one of the few asyncio database drivers that exists, notes in its FAQ "asyncpg uses asynchronous execution model and API, which is fundamentally incompatible with SQLAlchemy" [2], yet we know this is not true because SQLAlchemy works just fine with gevent and eventlet, with no architectural changes at all. Using libraries like SQLAlchemy or Django with a non-blocking IO, event-based model is commonplace. It's the "explicit" part of it that is hard, which is because of how asyncio is designed, without any mediation for code that doesn't publish "async / await" keywords in the middle.

So I finally just sat down to figure out how to use the underlying greenlet library (which we all know as the portable version of "Stackless Python") to bridge the gap between asyncio and blocking-style code, it's about 30 lines and I have SQLAlchemy working with an async front-end to asyncpg DBAPI as can be seen at [3] based on the proof of concept at [4]. I'm actually running the full py.test suite all inside the asyncio event loop and running asyncpg through SQLAlchemy's whole battery of thousands of tests, all of them written in purely blocking style, and there's not any need to add "async / await / yield / etc" anywhere except the very endpoints, that is, where the top function is called, and then down where we call into asyncpg directly, using a function called await_() that works just like the "await" keyword. Just no "async" function declaration.

A day later, someone took the same idea and got Flask to work in an asyncio event loop at [5]. The general idea of using greenlet in this way is also present at [6], so I won't be patenting this idea today as oremanj can claim prior art.

Using greenlet, there is no need to break out of the asyncio event loop at all, nor does it change the control flow of parallel coroutines within the loop. It uses greenlet's "switch", quite minimally, to bridge the gap between code that does not push out an "async/await" yield and code that does. There are no threadpools, no alternate event loops, no monkeypatching, just a few greenlet.switch() calls in the right spots. A slight performance decrease of about 15%, but in theory one would only be using asyncio if their application is expected to be IO bound in any case (which folks that know me know is another assertion I frequently doubt).

So to sum up, last week, libraries like Flask and SQLAlchemy were "fundamentally incompatible" with asyncio, and this week they are not. What's confusing me is that I'm not that smart and this is something all of the affected libraries should have been doing years ago, and really, while I know this is not going to happen, this should be *part of asyncio itself* or at least a very standard approach so that nobody has to assume asyncio means "rewrite all your library code".

To add an extra bonus, you can use this greenlet approach to have blocking-style functions right in the middle of your otherwise asyncio application. Which means this also is a potential solution to the "lazy-loading" problem. You have an asyncio app that does lots of asyncio to talk to microservices, but some functions are doing database work and they really would like to just work in a transaction, load some objects and access their attributes without worrying that a SQL statement can't be emitted. This approach makes that possible as well. ORM lazy loading with the asyncpg driver: [7] . Indeed, if you have a PostgreSQL SQLAlchemy application already written in blocking style, you can use this new extension and drop the entire application into the event loop and use the asyncpg driver, not too unlike using gevent except nothing is monkeypatched.

The recipe is simple and so far appears to be very effective. Using greenlet to manipulate the stack is of course "spooky" and I would assume Python devs may propose that this would lead to hard-to-debug conditions. I've used gevent and eventlet for many years and while they do produce some new issues, most of them relate to the fact that they use monkeypatching of existing modules and particularly around low level network drivers like pymysql. The actual stack moving around within business logic doesn't seem to produce any difficult new issues. Using plain asyncio has a lot of novel and confusing failure modes too. Using the little bit of "spookyness" of greenlet IMO is a lot less work than rewriting SQLAlchemy, Django ORM, Flask, urllib3, etc. from scratch and maintaining two codebases though.

[1] https://pypi.org/project/unasync/

[2] https://magicstack.github.io/asyncpg/current/faq.html#can-i-use-asyncpg-with-sqlalchemy-orm

[3] https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/2071

[4] https://gist.github.com/zzzeek/4e89ce6226826e7a8df13e1b573ad354

[5] https://twitter.com/miguelgrinberg/status/1279894131976921088

[6] https://github.com/oremanj/greenback

[7] https://gerrit.sqlalchemy.org/plugins/gitiles/sqlalchemy/sqlalchemy/+/refs/changes/71/2071/10/examples/asyncio/greenlet_orm.py

Thanks for posting this, Mike.

Vague claims of "framework X is faster because it's async" appear, impossible to confirm as it is unknown how much of their performance gains come from the "async" aspect and how much of it is that they happened to rewrite a new framework from scratch in a completely different way (hint: it's the latter).

These kind of claims are not specific to async vs. sync. They are all over the place for every two pieces of comparable technologies. While novice users might base their technology choice purely on such benchmarks, it's less of an issue for startups/tech companies.

That said, I agree with most of your points so far.

The asyncpg project, one of the few asyncio database drivers that exists, notes in its FAQ "asyncpg uses asynchronous execution model and API, which is fundamentally incompatible with SQLAlchemy" [2], yet we know this is not true because SQLAlchemy works just fine with gevent and eventlet, with no architectural changes at all.

But it is true. Making asynchronous network requests in asyncio requires async/await or using callbacks and it's not possible to do them, say, from __getattr__ (you mention this yourself). This is what that particular comment is about, nothing more. Using gevent and eventlet as examples in this particular context isn't helping you. Apologies for nitpicking, I know it's not the point of this discussion.

A day later, someone took the same idea and got Flask to work in an asyncio event loop at [5]. The general idea of using greenlet in this way is also present at [6], so I won't be patenting this idea today as oremanj can claim prior art.

Yes, this approach definitely works and I even did that in production myself a few years ago with https://github.com/1st1/greenio (it's terribly outdated now).

The recipe is simple and so far appears to be very effective.

This recipe was one of the reasons why I added loop.set_task_factory method to the spec, so that it's possible to implement this in an existing event loop like uvloop. But ultimately asyncio is flexible enough to let users use their own event loop which can be compatible with greenlets among other improvements.

Ultimately, asyncio will probably never ship with greenlets integration enabled by default, but we should definitely make it possible (if there are some limitations now). It doesn't seem to me that nested event loops are needed for this, right?

This recipe was one of the reasons why I added loop.set_task_factory method to the spec, so that it's possible to implement this in an existing event loop like uvloop. But ultimately asyncio is flexible enough to let users use their own event loop which can be compatible with greenlets among other improvements.

Right, when I sought to look at this, I know that my users want to use the regular event loop in asyncio or whatever system they are using.

Ultimately, asyncio will probably never ship with greenlets integration enabled by default, but we should definitely make it possible (if there are some limitations now). It doesn't seem to me that nested event loops are needed for this, right?

So right, the approach I came up with does not need nested event loops and it's my vague understanding that nesting event loops is more difficult to debug, because you have these two separate loops handing off to each other.

What is most striking about my recipe is that it never even leaves the default event loop. Originally my first iteration when I was trying to get it working, I had a separate thread going on, as it seemed intuitive that "of course you need a thread to bridge async and sync code" but then I erased the "Thread()" part around it and then it just worked anyway. Like it's simple enough that shipping this as a third party library is almost not even worth it, you can just drop this in wherever. If different libraries had their own drop-in of this, they would even work together. greenlet is really like totally cheating.

the philosophical thing here is, usually in my many twitter debates on the subject, folks get into how they like the explicit async and await keywords and they like that IO is explicit. So I'm seeking to keep these people happy and give then "async def execute(sql)", and use an async DB driver, but then the library that these folks are using is internally not actually explicit IO. But they don't have to see that, I've done all the work of the "implicit IO" stuff and in a library it's not the same kind of problem anyway. I think this is a really critical technique to have so that libraries that mediate between a user-facing facade and TCP based backends no longer have to make a hard choice about if they are to support sync vs. async (or async with an optional sync facade around it).

I think this is a really critical technique to have so that libraries that mediate between a user-facing facade and TCP based backends no longer have to make a hard choice about if they are to support sync vs. async (or async with an optional sync facade around it).

If this works for such a big and elaborate framework as SQLA, we can definitely highlight this as a valid approach and even add a link to a blog post from the docs. We'll need to add an asyncio specific FAQ page for that or something similar.

Another approach, which would probably be a nonstarter for SQLA, is to use async/await for literally everything internally, and provide a tiny synchronous facade on top. Funny thing you don't even need an event loop for that, just the basic understanding of how coroutines work internally. I used this to create the edgedb-python package which has both sync and async first-class support with one code base. Sync is even faster there in simple throughput benchmarks (as expected).

yes so if you have async/await all internal, are you saying you can make that work for synchronous code *without* running the event loop? that is, some kind of container that just does the right thing? my concern with that would still be performance. When asyncio was based on yield and exception throws, that was a lot of overhead to add to functions and that was what my performance testing some years back showed. w/ async/await I'm sure things have been optimized, but in general when i have function a() -> b() -> c(), I am trying to iron as much Python overhead as I possibly can out of that and I'd be concerned that the machinery to work through async/await would add latency. additionally if it was async/await internally but then i need to access the majority of Python DBAPIs that are sync, I need a thread pool anyway, right? which is also another big barrier to jump over.

It seems you were involved with urllib3's approach to use a code rewriter rather than a runtime approach based on the discussion at urllib3/urllib3#1323 , but it's not clear if Python 2 compatibility was the only factor or if the concern of "writing a giant shim" was also.

Yeah, writing a trivial "event loop" to drive actually-synchronous code is easy. Try it out:

-----

async def f():
    print("hi from f()")
    await g()

async def g():
    print("hi from g()")

# This is our event loop:
coro = f()
try:
    coro.send(None)
except StopIteration:
    pass

I guess there's technically some overhead, but it's tiny.

I think dropping 'await' syntax has two major downsides:

Downside 1: 'await' shows you where context switches can happen: As we know, writing correct thread-safe code is mind-bendingly hard, because data can change underneath your feet at any moment. With async/await, things are much easier to reason about, because any span of code that doesn't contain an 'await' is automatically atomic:

---

async def task1():
    # These two assignments happen atomically, so it's impossible for
    # another task to see 'someobj' in an inconsistent state.
    someobj.a = 1
    someobj.b = 2

This applies to all basic operations like __getitem__ and __setitem__, arithmetic, etc. -- in the async/await world, any combination of these is automatically atomic.

With greenlets OTOH, it becomes possible for another task to observe someobj.a == 1 without someobj.b == 2, in case someobj.__setattr__ internally invoked an await_(). Any operation can potentially invoke a context switch. So debugging greenlets code is roughly as hard as debugging full-on multithreaded code, and much harder than debugging async/await code.

This first downside has been widely discussed (e.g. Glyph's "unyielding" blog post), but I think the other downside is more important:

'await' shows where cancellation can happen: Synchronous libraries don't have a concept of cancellation. OTOH async libraries *are* expected to handle cancellation cleanly and correctly. This is *not* trivial. With your sqlalchemy+greenlets code, you've introduced probably hundreds of extra unwinding paths that you've never tested or probably even thought about. How confident are you that they all unwind correctly (e.g. without corrupting sqlalchemy's internal state)? How do you plan to even find them, given that you can't see the cancellation points in your code? How can your users tell which operations could raise a cancelled exception?

AFAICT you can't reasonably build systems that handle cancellation correctly without some explicit mechanism to track where the cancellation can happen. There's a ton of prior art here and you see this conclusion over and over.

tl;dr: I think switching from async/await -> greenlets would make it much easier to write programs that are 90% correct, and much harder to write programs that are 100% correct. That might be a good tradeoff in some situations, but it's a lot more complicated than it seems.

Yeah, writing a trivial "event loop" to drive actually-synchronous code is easy. Try it out:

This is exactly the approach I used in edgedb-python.

I guess there's technically some overhead, but it's tiny.

Correct, the overhead isn't even detectable in microbenchmarks. In most async programs regular function calls dominate awaits by a few factors of magnitude.

I think dropping 'await' syntax has two major downsides:

For the extra context, in the case of using this approach for something like edgedb-python these downsides don't really apply, because we're adapting async/await implementation to be sync. The async/await code can handle cancellation etc. whereas the sync code only needs to support the general protocol parsing flow. FWIW I don't think it would be possible to apply my approach to SQLA without a very invasive rewrite, which isn't probably worth it.

tl;dr: I think switching from async/await -> greenlets would make it much easier to write programs that are 90% correct, and much harder to write programs that are 100% correct. That might be a good tradeoff in some situations, but it's a lot more complicated than it seems.

Yeah, this sums up my opinion on this topic.

Also, having spent a couple of years writing and debugging big and small greenlet-heavy code bases I wouldn't want to touch them ever again. Your mileage may vary, Mike.

With greenlets OTOH, it becomes possible for another task to observe someobj.a == 1 without someobj.b == 2, in case someobj.__setattr__ internally invoked an await_(). Any operation can potentially invoke a context switch. So debugging greenlets code is roughly as hard as debugging full-on multithreaded code, and much harder than debugging async/await code.

I would invite you to look more closely at my approach. The situation you describe above applies to a library like gevent, where IO means a context switch that can go anywhere. My small recipe never breaks out of the asyncio event loop, and it only context switches within the scope of a single coroutine, not to any arbitrary coroutine. So I don't think the above issue applies.

Additionally, we are here talking about *libraries* that are independently developed and tested distinct from end-user code. If there's a bug in SQLAlchemy, the end user isn't the person debugging that. arguments over "is async or sync easier to debug" are IMO pretty subjective and at this point they are not relevant to what sync-based libraries should be doing.

Do we have to allow people writing asyncio.run(coro())? If they can just as well write await coro() why not recommend that as the fix, instead of installing nest_asyncio?

They can only await coro() at the top-level, but if we recommend doing that in a function down the stack, now they have to convert pretty much all their code base to async. I think it would be nice to nest event loops not only in the Jupyter context, but for anyone willing to provide a blocking API from an async API.

But providing a simple way to run an asynchronous API synchronously is exactly the antipattern I'm trying to avoid here. If an API exists in async and sync form (the latter using asyncio.run() or the former), we want to make sure that when the user is already in async code they don't call the sync form out of laziness.

The two cases where we do want to allow using the sync form are: (1) in legacy sync code, where no loop is running; or (2) in situations like Jupyter, where there is a "dormant" event loop that is safe to ignore with a recursive asyncio.run() call is made. I like the idea of making the "dormant" loop advertise that it is okay with recursive asyncio.run() calls, either using a context manager.

@davidbrochart

Do we have to allow people writing asyncio.run(coro())? If they can just as well write await coro() why not recommend that as the fix, instead of installing nest_asyncio?

I think @davidbrochart replied to this. We are rarely talking about top-level asyncio.run, in which case, yes, top level await is the answer.

The proposal to allow writing with loop.subloop() as subloop: await subloop.create_task(coro()) is intriguing; it makes it clear that there is something funny with the event loop, but it fails to let people use asyncio.run(coro()), which they presumably copy-paste from one of the gazillion tutorials

Oh, I was not suggesting the inner code would have access to subloop. I was suggesting:


foo() # indirectly call asyncio.run() fails.

with some_decorator():
    foo() # indirectly calling asyncio.run() works.

We can just "isolate" the code that may run subloops. Typically in IPython/jupyter, the only place we would have this decorator is around whatever will call exec:

async def handle_user_request(code):
   with some_contextmanager():
        self.IPython.exec_user_code(code)

With this code could contain an asyncio.run, but we are sure that anywhere else in Jupyter we do not nest eventloops.

some_contextmanager would likely set a contextvar that let the next (second) usage of asyncio.run to not fail (semantics of weak loop, or concurrent loop to be allowed TBD). This (second) nested asyncio.run create a new context, so a third nested asyncio.run would fail (unless itself in an unprotected section with another some_contextmanager). My proposal would be that some_contextmanager() be a method on the current loop so that only code that has access to the current loop can create an unprotected section. And in the same way that trio has nurseries, you must pass this to your callee if you want them to decide to create an unprotected section.

To your second message

But providing a simple way to run an asynchronous [...]

I think you are basing your reflexion on case where the user/developper can be in control of most of the stack – and/or it's their job to understand and modify it – which is more common in businesses and applications, less in libraries and the Scientific ecosystem. There are many cases where this is untrue, and cases where you do not know wether the API is going to be async or sync down(or up) the line, or worse where you know you have a sync call between two async.

Same code Sync and Async Sandwich with anyway Sync in the middle.

Say you are doing machine learning on images in Jupyter.
You are likely using numpy array-like code.

import skimage
import sklearn
img = skimage.read('holy_grail.png')
sklean.detect_objects(img)

All good, your teacher is happy. sklean.detect_objects(img) is 100% Sync cause img[...] is sync, and you are a student on your laptop.

Now you get an image from the VLT, damn now it's a petabyte.

import skimage
import sklearn
img = skimage.read("s3://bucket/bucket/bucket.zarr")
sklean.detect_objects(img) # ???

Well now img[...] would like to do async stuff cause img[0:100, ..], and img[101:200,...] are on different servers and I can get them concurrently it's automatically using zarr (compressed, chunked, N-dimensional arrays) and compute concurently for independant chunks with dask (distributed computing) on your cluster cause your admin is awesome and enabled the Dask backend.

Great thing about Dask, and Zarr is they both handle one side of the numpy API, and both want async. Dask replace the computation, so calls get/setitem, zarr gets the data so implement get/setitem. Feel free to replace with with cupy if you want GPU, or ray for single machine parallelism. And here even if both want async, they can't know the other will be async. Well they are stuck. They don't know either exactly how they will interact with each other cause the algorithm is in sklearn/skimage, which itself tries not to care wether up/down is async, it just want to have array of pixel and do stuff with it in some order.

Sure I could make await async_detect_objects(img)... but anyway img[...] is def not async def it will be sync here anyway. Or the user pass a custom callback that is sync. Those algo have been written for decades, so even if the new ones are async def anyway and we now have async def getitem, I might still get called by (or with) old sync ones.

Even if we – as maintainer – would like to use async all the way down, in practice we hit so many layers and places where we can't use async, or don't know if we will get async we have to nest loops. Cause we'd like the user to keep the same code for both use case. (also "Although practicality beats purity").

Jupyter (scientific users are not here to code).

Many of Jupyter Users are also not programers, their job is not to code, it's to produce science. They can't afford to learn about async, it's not their job. They are not lazy not learning about it, because often the tradeoff is not worth it anyway.

They are not going to serve 1M page. Most code is going to be ran once. I mean we are sometime talking about code that take you 15 second to write, and runs from 1-2 seconds and gets modified/deleted. Reaching for async everywhere means taking 30 second to write it, maybe more as it cause Errors the first 2 times, and googling on stack overflow. The result will now take only 0.5 sec to display. But for that scientist that's ~95% worse.

Plus now they have to context switch between the "Mutations within a furin consensus sequence block proteolytic release of ectodysplasin-A", and "how do you properly reuse an httpx.AsyncClient". Now I personally find both question fascinating, but having seen some folks struggling with dicts, I don't think they are ready to tackle the details async/await. Though I still they may help to cure cancer, and that Async/Await are not going to help with that for some of them.

For those interested in both above question, I'd like for them to be able to focus on the Furin cleavage site during the day, and do science, and httpx during the night, if they like to program, but not both at the same time.

For those scientist the metrics of "clean/fast" code is completely different and they love python because it's pseudo code, and it just works™. If it's too slow, they know where my office is.

Maintainers for Science

I think the Scientific Python stack developers are doing their best to make things work out of the box as well as possible without having users to look under the hood, and we would be grateful for a way to at least locally enable a weaker concurrency form (zarr perfect example as it's a leaf and we just want getitem to have local concurency).

Now what would be even greater, is that if all the ingredient of the sandwich were gluten free, to not have to package each individually and have them all running in the same loop. It will just take some reallllllly long time to get there.

We also do our best to put warnings when people do thing in non optimal way, and we love when a grad student reach to us and use the right API and suddenly have their code run in 12 min instead of 12days (true story), and do completely understand your POV. We just have a sometime a realllly different use case.

Sorry for this long message, if you read me in full (whoever you are), I owe you a drink next we meet at a conference.

@fried

FWIW for a different view on nesting event loops, see @fried's talk from PyCon 2022: https://youtu.be/XW7yv6HuWTE. (And yes, I'm slowly working through my backlog, and I will eventually get to this.)

Anyway, you now definitely owe me a drink. :-)

I think I get your point about scientists, best they don't have to know about async at all. So maybe it's Jupyter's fault for having an event loop running already. If what you say is roughly true, top-level await is addictive candy for CS nerds, but rarely useful for scientists, and causes them pain (which nest_asyncio solves with a big hammer).

But if we accept that Jupyter wants to do it this way, your proposal of having a context manager that sets something like a context var to communicate to asyncio.run() that it should ignore the existing loop makes sense.

We very much have to debate whether it's better in that case for asyncio.run() to reuse the existing loop or whether it should create a new loop.

Creating a new loop is probably simpler to implement in run(), because it would (mostly) just mean that it can ignore the check for "is a loop already running" and everything else remains the same. A possible exception would be the set_event_loop() call and the logic around that -- it might have to save/restore the "current" event loop.

The semantics of creating a new loop are also simpler: the outer loop is just as blocked as when you call some CPU-consuming function. And no promises are broken except the "don't do blocking I/O without using await".

Reusing the loop would make the implementation slightly more complicated (I think), but the real downside is that it does break the promise I take most seriously -- in an asyncio world, other tasks and callbacks are guaranteed only to run at await points. OTOH it consumes fewer resources (there's only one event loop object), and tasks/callbacks in scenarios that don't care about the strong guarantee will not be blocked.

All in all I think I'd just approve a properly done PR that implements this idea if it creates a new loop, but I'd be wary of a version reusing the loop, so if you feel strongly that the latter is better, you'll have to show a very clear scenario here.

@zzzeek

I beg you not to get hung up on Jupyter as the only use case prompting this requirement. The scenario described by @zzzeek above is also important; an existing library written in blocking code which can load drivers to interact with network resources. Naturally the writers of those drivers want to be able to use asyncio under the covers. But the library developers don't want to have to rewrite their entire library to support that. So you end up with red/blue delineation, not only at the function level but at the library level.

So many words. Does anyone have any code to contribute that satisfies various requirements?

I actually thought I needed this to prevent code duplication within decorators that wrap both async and sync code.

But inspired by @njsmith I figured out a solution which I documented in stackoverflow.

@CarliJoy

Thank you @CarliJoy for sharing your solution.

@gvanrossum I would like to close this issue. IMHO, I don't see us adding nested event loops due to the added complexity and overhead. I think reasoning about event loops is difficult enough with one loop, and nested event loops would contribute to even more unpredictable behavior in a program.

I support a low level asyncio-based library which I must be able to expose through a 3rd party non-async library (maybe even multiple such layers), all of which needs to be executable from within Jupyter (where cells are run in an async loop). The only solution I've found has been nest_asyncio. I would welcome some official Python documentation/examples explaining what I'm supposed to do or I request some kind of endorsement of nest_asyncio.

As an aside, I just discovered that Ewald de Wit, the author and maintainer of nest_asyncio, has just passed away. RIP.

@qci-amos

@qci-amos The latest library I learnt about that could help you is greenback.

@CarliJoy

So 3 years after my previous comment, it's back to square one:

Running nest_asyncio.apply() under gunicorn gives an exception immediately:

  File "/lib/python3.12/site-packages/nest_asyncio.py", line 193, in _patch_loop
    raise ValueError('Can\'t patch loop of type %s' % type(loop))
ValueError: Can't patch loop of type <class 'uvloop.Loop'>

This is not too surprising TBH, considered the implementation relies on pretty hacky monkey patching.

Unless the project gets forked/re-owned, this will not be fixed
The greenback alternative seems to provide a useful building block, but for how long and with what reliability ? From greenback's own doc:

The approach required deep architecture-specific magic and elicited a number of subtle bugs, but those have generally been worked out in the years since the first release in 2006, such that the package is now considered pretty stable.

Although I'm glad bugs got ironed out, it does not fill me with confidence, especially considered the general amount of change in recent CPython versions (JIT in 3.13 etc).

@CarliJoy 's SO post is interesting to remove duplication in decorator code, but brings nothing when it comes to the backward compat use case (my initial comment). Taking the place of the event loop and using .send(None) obviously fails pretty quickly on even a basic await asyncio.sleep(), which is not suprising as the asyncio event loop does a bit more than nothing at all :)
The only solution that does not rely on deep low-level hacks or monkey patching I found was to spin up a thread for every re-entrant call, and wait for the thread to finish. This is not satisfactory since the perf cost is very high and requires thread safety in the code. On top of that, the lib I'm working on uses thread local values, so running some code in another thread can trigger a re-connection over SSH, and using a thread pool means these extra connections become somewhat immortal.

So unless I missed anything the problem is still opened: there is no reasonable way to provide a sync API shim for backward compat on top of an async API using asyncio.

@willingc Should I open a new issue specifically for that backward compat use case ? I'm not particularly attached to any implementation detail (nested event loop or otherwise) so it's fine to me that this specific thread got closed on those grounds:

IMHO, I don't see us adding nested event loops due to the added complexity and overhead

However, the actual issue I'm facing is still very much unsolved, and closing the thread is unfortunately not going to change that, and it will not help discoverability for other people having the same problem. If greenback is the endorsed way to go then so be it, but I'll be hesitant until I see an official recommendation to use it.

@douglas-raillard-arm

@douglas-raillard-arm I think opening a new issue makes sense if you focus on the specific issue that you are facing. Please focus more on the issue and less on how to solve the issue. Including a small executable example would be helpful too. Thanks!

I think I get your point about scientists, best they don't have to know about async at all. So maybe it's Jupyter's fault for having an event loop running already. If what you say is roughly true, top-level await is addictive candy for CS nerds, but rarely useful for scientists, and causes them pain (which nest_asyncio solves with a big hammer).

Q: Is this a discussion about function colouring, or is it a discussion about interactive vs. non-interactive use?

Last time I tried to push a similar boulder up a similar hill, it was definitely about function colouring:

https://github.com/gsmecher/tworoutine

This approach occupied a losing position, for all the reasons discussed above. (Plus it muddies a distinction between async and non-async that's at least conceptually clear, if inconvenient).

If the pain point is interactive vs. non-interactive use, rather than function colouring, maybe a slightly different attempt is worthwhile:

https://github.com/gsmecher/awaitless

I'm hoping this reduces the pain points associated with asyncio in a scientific / interactive / instrumentation context. Maybe if we can do that, async will seem a little less noxious to our non-CS users.

Navigation Menu

Repository navigation

asyncio: nested event loop #66435

Linked PRs

24 remaining items

Same code Sync and Async Sandwich with anyway Sync in the middle.

Jupyter (scientific users are not here to code).

Maintainers for Science

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

asyncio: nested event loop #66435

Description

Linked PRs

Activity

djarb commented on Aug 20, 2014

gvanrossum commented on Aug 25, 2014

djarb commented on Aug 28, 2014

RokasKrku commented on Jul 15, 2017

zzzeek commented on Jul 6, 2020

1st1 commented on Jul 6, 2020

zzzeek commented on Jul 6, 2020

1st1 commented on Jul 6, 2020

zzzeek commented on Jul 6, 2020

njsmith commented on Jul 7, 2020

1st1 commented on Jul 7, 2020

zzzeek commented on Jul 7, 2020

24 remaining items

davidbrochart commented on May 2, 2023

gvanrossum commented on May 2, 2023

Carreau commented on May 2, 2023

Same code Sync and Async Sandwich with anyway Sync in the middle.

Jupyter (scientific users are not here to code).

Maintainers for Science

gvanrossum commented on May 8, 2023

gvanrossum commented on May 17, 2023

tomkcook commented on Oct 12, 2023

gvanrossum commented on Oct 12, 2023

CarliJoy commented on Mar 16, 2024

willingc commented on Mar 16, 2024

qci-amos commented on Mar 18, 2024

davidbrochart commented on Mar 18, 2024

douglas-raillard-arm commented on Apr 26, 2024

willingc commented on Apr 28, 2024

gsmecher commented on Aug 15, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions