ABC for Futures

@gvanrossum

I'm experimenting with implementing an asyncio-compatible event loop on top of libuv with Cython. It would be great if there is a way to provide a custom Future implementation. I was thinking about adding loop.create_future() (instead of Future(loop=loop)) API method, as well as an ABC, so that functions like ensure_future would recognize custom Futures. @gvanrossum @Haypo what do you think about this?

@gvanrossum

Hm... Why can't your new loop reuse the existing Future class? Or subclass
it?

On Sun, Nov 1, 2015 at 4:45 PM, Yury Selivanov notifications@github.com
wrote:

I'm experimenting with implementing an asyncio-compatible event loop on
top of libuv with Cython. It would be great if there is a way to provide a
custom Future implementation. I was thinking about adding
loop.create_future() (instead of Future(loop=loop)) API method, as well
as an ABC, so that functions like ensure_future would recognize custom
Futures. @gvanrossum https://github.com/gvanrossum @Haypo
https://github.com/haypo what do you think about this?

—
Reply to this email directly or view it on GitHub
#282.

--Guido van Rossum (python.org/~guido)

I can reuse the existing Future (but I can't derive extension types from Python classes in Cython). I just wanted to make a little bit faster Future, and, especially, Task classes.

Maybe I'm trying to micro-optimize here.

Hack asyncio in a local branch, come with a microbenchmark and then we can discuss ;-) Once I implemented a free list for Task. The gain was null, so I gave up.

@Haypo

@Haypo Yeah, I'm closing this issue for now. I'll come back with a microbenchmark if there is any difference.

@gvanrossum

@gvanrossum @Haypo @asvetlov So now I have benchmarks ;)

Long story short, here is an implementation of an asyncio-compatible event loop on top of libuv: https://github.com/1st1/uvloop. It's an early preview, however, at this point enough is implemented to run various benchmarks. I think that I got most of the low-level implementation right, now I need to add more tests and implement the missing APIs.

For benchmarks I used the ones that David Beazley used for his curio project.

Echo server using low-level `loop.sock_recv` / `loop.sock_sendall`

Results, Python 3.5, best of 3, 600,000 ~1kb messages sent in 3 parallel processes to a single server process:

asyncio with built-in event loop

600000 in 43.61588096618652
13756.457205694267 requests/sec

asyncio with uvloop

600000 in 17.48739981651306
34310.418146523414 requests/sec

gevent (Python 2.7)

600000 in 15.572859048843384
38528.57064448694 requests/sec

curio

600000 in 21.155117988586426
28361.93115650364 requests/sec

Echo server using asyncio streams

Results, Python 3.5, best of 3, 600,000 ~1kb messages sent in 3 parallel processes to a single server process:

asyncio with built-in event loop

600000 in 28.544986963272095
21019.452584511615 requests/sec

asyncio with uvloop

600000 in 14.964998960494995
40093.55440544273 requests/sec

Analysis

Bottom line, asyncio with uvloop is about ~1.85-2.5x faster.

I was initially surprised to see that the streams server was so fast, since there is quite a bit of Python code in asyncio/streams.py. The explanation is that in uvloop, transports are implemented directly on top of libuv streaming abstractions. All buffering and low-level IO is managed by libuv directly and efficiently.

Future objects in C

The above results were produced with unmodified asyncio (stock Future, stock Task, stock streams.py).

In order to see if I can further improve the performance, I implemented Futures in pure C: uvloop/future.c. In microbenchmarks, C version is 2x faster than asyncio/futures.py.

Here's the result of running sock_recv / sock_sendall benchmark with C Future:

600000 in 12.447252035140991
48203.41054443859 requests/sec

Another ~25% improvement (3.5x faster than vanilla asyncio, 1.25x faster than gevent), absolute win! I think it would make sense to implement this as well. Below is a list of modifications to asyncio that we would need:

Future has to be an ABCMeta class, so that existing isinstance checks will continue working with new futures.
isinstance checks are very slow, and are visible when profiling, especially in Task._step. I'm thinking about something like loop.is_future method, that can be implemented very efficiently. Alternatively, we can just use duck-typing (in Task._step) by substituting isinstance(result, Future) with hasattr(result, 'add_done_callbacks')
In order to make it possible to switch all code to new futures, we'll need loop.create_future (similar to the existing loop.create_task).
The existing Future class has some methods that shouldn't be methods at all, for instance _set_result_unless_cancelled or _copy_state. Those only use Future's public API, and can easily be factored out into separate helper functions.
Future._loop and Future._blocking have to become public API (get_loop(), was_yielded()?).

Exciting -- but can you find some collaborators to help with this? I can't take this on. :-(

@gvanrossum

@gvanrossum To be clear, I'm not proposing to merge uvloop into asyncio, it should stay as a separate project for a while, before we can even consider that. I want to do some modifications to the asyncio.Future - mainly inheriting it from ABCMeta, stripping some of its undocumented APIs, and adding loop.create_future() method. If you and @Haypo can give me a greenlight to do that, I'd be happy to create a PR.

Do the modifications also benefit plain vanilla asyncio? By how much? I am hesitant to change any public APIs; in particular requiring people to call loop.create_future() seems onerous. Would it be possible to gain a similar speedup just by rewriting some of asyncio.Future's internals, without any public API changes?

I am hesitant to change any public APIs; in particular requiring people to
call loop.create_future() seems onerous.

Calling asyncio.Future(loop=loop) isn't much different from loop.create_future(). The latter form is less error prone, since the future will always be attached to the correct loop. I, myself, always pass the loop argument manually. And, by the way, nothing will prevent people from using asyncio.Future, as loops must work with any compatible implementation.

In my experience (and I may be wrong here), people don't create futures by hand often. Instead they use asyncio through frameworks like aiohttp, which simply expose some awaitables.

Would it be possible to gain a similar speedup just by rewriting some of
asyncio.Future's internals, without any public API changes?

It's a bit tricky. Essentially, my C implementation of the Future class is very similar to the Python one. Here are some reasons why it's faster:

Since it's a C object, there is no __dict__ lookups. And being implemented in C, we don't have opcodes/ceval overhead.
__iter__ and __await__ return the future object itself, and the future implements a __next__ slot in a very straightforward way. Python version, in contrast, creates a generator on each call to __await__. In Python it's a better strategy, but in C my approach is much faster.

Speaking about new APIs, I don't think we need much. Many of the ones I listed above are very nice to have, but let me iterate through them and highlight the most important ones:

[important] The main one is making asyncio.Future an ABCMeta. This will allow me to have a custom Future implementation and all existing asyncio code (ensure_future etc) will accept it.
[important] Future._set_result_unless_cancelled and Future._copy_state need to be factored out to standalone functions. This shouldn't be a problem - they aren't part of any public API, they themselves use only public API, and they are used in just a few places in asyncio. To preserve absolute backwards compatibility we can keep the methods, but modify the code that uses them to use standalone functions.
[very nice to have] loop.create_future() method. I think it'd be quite handy to use. It will improve performance in any code that uses it and is running with uvloop or any other optimized event loop.
[optional] Make public APIs for Future._blocking and Future._loop. These I can easily implement in C as getters/setters, so this is really optional.
[optional] I'd want to slightly tweak Task._step and Task._wakeup. The former will use hasattr(result, 'add_done_callback') instead of isinstance(result, Future) (duck typing). And it would be nice to change the latter to call self._step slightly different (I can explain in the patch why). This change will also slightly improve performance of asyncio.Future.

I'd be very willing to contribute my Future implementation to asyncio and CPython. But I'm not sure what's the timeframe here is going to be. Everything except loop.create_future is very lightweight in terms of API changes, and loop.create_future is something that I genially believe would be a great addition regardless of its performance benefits.

Hum, I'm not sure that we want an ABC here. Yury proposes a C accelerator module for asyncio. I really like this idea, I had the same idea many times (I tried to implement a free list for the Task class, but I didn't see any difference in performance). I know that writing the same code with the Python C API makes the code faster (I don't understand why, it's just a fact :-)).

IMHO we need an _asyncio module which implements some functions and classes of asyncio, but keep the Python implementation for other Python VMs (PyPy & cie). And then use the classic "try: from _asyncio import Future except ImportError: Future = PyFuture".

"Calling asyncio.Future(loop=loop) isn't much different from loop.create_future(). The latter form is less error prone, since the future will always be attached to the correct loop."

We can consider this as a different issue. For me, it's a an API enhancement to make the loop "parameter" mandatory (and it might be a little bit faster, no need to get asyncio.get_event_loop()). The drawback is that you have to pass loop to a lot of functions. I don't know if users like the implicit loop parameter. Maybe passing explicitly the loop parameter matters more when you have multiple event loops running in different threads? (but asyncio.get_event_loop() handles this use case thanks to thread local storage).

I'm ok to add loop.create_future() and try to use it in asyncio core, but I'm not sure yet that we really must to replace all Future(loop=loop) with loop.create_future(). As the addition of loop.create_task(), it also creates a problem with backward compatibility. To write code working with asyncio of Python 3.4.0 stdlib, create_task() and create_future() cannot be used... as await/async keywords...

Very exciting results!

1 Making asyncio.Future an ABCMeta.

Agree.

2 Future._set_result_unless_cancelled and Future._copy_state need to be factored out to standalone functions.

I see no problems with modifying private part if asyncio on our own.

3 loop.create_future() method.

Totally support the idea. loop.create_future() looks like loop.create_task() and has similar motivation.

4 Make public APIs for Future._blocking and Future._loop.

I doubt we need it public, especially _blocking attribute. There is nothing bad with public .loop though.

5 I'd want to slightly tweak Task._step and Task._wakeup. The former will use hasattr(result, 'add_done_callback') instead of isinstance(result, Future) (duck typing).

I don't like duck typing in general but in this case we can live with it if the change improve performance significantly.

@Haypo

@Haypo I think Future implementation may be coupled event loop, so having the single _asyncio.Future may not satisfy all requirements. loop.create_future() looks much more clean and native.

About explicit/implicit loop. I've found the former extremely important, all my public libraries and private code use explicit loop only. It helps with code and tests isolation very well and don't bring too much work with passing loop parameter everywhere.
Mixing loops (e.g. explicit and implicit) leads to hard-to-fix errors. I consider libraries that doesn't accept explicit loop as not ready to be used.

Inexperienced users likes implicit loop, yes -- until they shoot own leg by the technique. After that they switches to explicit one. I saw it several times.

About backward compatibility problems. .create_future() is not mandatory, developers still may use Future(loop=loop) call.

Library authors may adopt their code to use loop.create_future() if available. That's easy and don't require a lot of modifications. Say, aiohttp has 6 places with explicit future creation only, the same is true for all my other libraries.

Yes, aiohttp like any other asyncio library creates a lot of future instances in test suite but as Future() is still supported the test suite doesn't require any change.

I think 25% speedup merits adding new optional method to event loop. I definitely appreciate changes which makes aiohttp-based applications several times faster.

IMHO we need an _asyncio module which implements some functions and classes of asyncio, but keep the Python implementation for other Python VMs (PyPy & cie). And then use the classic "try: from _asyncio import Future except ImportError: Future = PyFuture".

Yes, I want _asyncio too. OTOH I don't want to wait for it for many months. It would be great if I can develop a high performance Future implementation as part of uvloop project, and once it's stable, I'll contribute it to asyncio.

Without the ABC thing, though, I can't do that. I don't see how having this ABC can cause any problems.

[..] As the addition of loop.create_task(), it also creates a problem with backward compatibility. [..]

I don't understand this argument. By this logic we can't ever add any new APIs. Again, the old asyncio.Future(loop=loop) will continue working forever. If you want to use create_future - use it, we'll document when it is available.

Inexperienced users likes implicit loop, yes -- until they shoot own leg by the technique. After that they switches to explicit one. I saw it several times.

Yes, this reflects my experience too.

About backward compatibility problems. .create_future() is not mandatory, developers still may use Future(loop=loop) call.

Exactly!

I suggest to start with a pull request to add create_future() which would simply instanciate Future(). So at least, you can manually override the class method in your implementation of the event loop.

Later, we can discuss adding a future factory API, and implement _asyncio in C.

I suggest to start with a pull request to add create_future() which would simply instanciate Future(). So at least, you can manually override the class method in your implementation of the event loop.

Great! I still need the ABCMeta though, otherwise it won't work.

Later, we can discuss adding a future factory API, and implement _asyncio in C.

I'm down to contribute my C Future before 3.5.2.

Way too much text to read...

Here's what I think would be acceptable.

Hold back until 3.5.1 is released (rc1 Nov 22, final Dec 6, according to
PEP 478); we need other issues fixed first!
There can be an AbstractFuture which can be the base class for Future.
All places in asyncio that currently check isinstance(f, Future) can be
changed to isinstance(f, AbstractFuture).
There can be a loop.create_future() method, and all code in asyncio can
call it instead of Future().
I'm also okay with removing those private methods. I expect some fallout
from this -- other libraries may use them. We need to start a PR campaign.
We could also keep them but put a warning in the code until 3.6.
BUT: users must still be able to manually instantiate Future() or Task()
and as long as they aren't using your libuv-based loop their code should
work.
Also, user code that checks isinstance(f, Future) or isinstance(f, Task)
must not break (unless using your libuv-asyncio). And Task must continue to
inherit from Future (not just AbstractFuture).

An alternative would be to skip the AbstractFuture -- if we have an
_asyncio extension module, it can define a Future class and that should be
the only one you need, right?

Can this AbstractFuture class be an ABCMeta? I don't think I can derive a C class from a Python class.

Sure. Then you plan to use register(), right? Have you timed the speed of isinstance() checks involving ABCMeta and registration()? Isn't that expensive?

@1st1

Wow, congratulations @1st1, it's very exciting, I'm very interested in to test that as soon as possible ;-)

Some remarks:

To evaluate how many open source projects use a part of AsyncIO API, we should use a search engine like this: http://nullege.com/codes/search?cq=asyncio.Future or via github search engine itself. Maybe the proposed changes don't impact too much existing open source code.
If the changes aren't too invasive (it doesn't seem to be), it shouldn't too hard to propose pull requests to take this benefit. AsyncIO users could help to propagate that on libraries.
All AsyncIO libraries I use have an optional parameter to pass explicit event loop: It shouldn't be an issue for uvloop.
I'll benchmark that with FrameworkBenchmarks as soon as possible. You can count on me to integrate that officially in the benchmarks suite when it will be ready.

Can this issue be closed now?

Can this issue be closed now?

Yeah, I don't think we need it open anymore. Thank you very much for the code reviews!

Small offtop:
AFAIK, libuv also support async filesystem calls. Can this be implemented in asyncio/loop/3rd/whatever with nice await with and some pathlib awaitable coroutines? Then «not good» executors for filesystem calls can be dropped 👏

@pohmelie: that would be a question for the uvloop project: https://github.com/1st1/uvloop

Navigation Menu

Repository navigation

ABC for Futures #282

Echo server using low-level `loop.sock_recv` / `loop.sock_sendall`

Echo server using asyncio streams

Analysis

Future objects in C

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

ABC for Futures #282

Description

Activity

gvanrossum commented on Nov 2, 2015

1st1 commented on Nov 2, 2015

vstinner commented on Nov 2, 2015

1st1 commented on Nov 3, 2015

1st1 commented on Nov 12, 2015

Echo server using low-level loop.sock_recv / loop.sock_sendall

Echo server using asyncio streams

Analysis

Future objects in C

gvanrossum commented on Nov 12, 2015

1st1 commented on Nov 12, 2015

gvanrossum commented on Nov 12, 2015

1st1 commented on Nov 12, 2015

vstinner commented on Nov 12, 2015

asvetlov commented on Nov 12, 2015

asvetlov commented on Nov 12, 2015

1st1 commented on Nov 12, 2015

vstinner commented on Nov 12, 2015

1st1 commented on Nov 12, 2015

gvanrossum commented on Nov 12, 2015

1st1 commented on Nov 12, 2015

gvanrossum commented on Nov 12, 2015

ludovic-gasc commented on Nov 12, 2015

gvanrossum commented on Nov 17, 2015

1st1 commented on Nov 17, 2015

pohmelie commented on Nov 17, 2015

gvanrossum commented on Nov 17, 2015

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Echo server using low-level `loop.sock_recv` / `loop.sock_sendall`