Repository navigation
asyncio.as_completed() raises TypeError when the first supplied parameter is a generator that yields awaitables #88342
Description
Activity
alexdelorenzo commented
on May 19, 2021 alexdelorenzomannequinMannequinAuthorAccording to the documentation, asyncio.as_completed() takes a positional argument, aws, as an iterable of awaitables[1]:
asyncio.as_completed(aws, *, loop=None, timeout=None) Run awaitable objects in the aws iterable concurrently.
As seen in the attached as_completed_gen.py file, built-in containers like lists, and iterators over them, are accepted by as_completed() as the first parameter without raising an error.
However, as_completed() raises TypeError if it is called with an iterable of awaitables that is also a generator. There are examples of this behavior in as_completed_gen.py, but here is a short example using a generator expression in the main() coroutine function:
from asyncio import run, as_completed async def example(): pass async def main(): coros = (example() for _ in range(10)) for coro in as_completed(coros): # raises TypeError await coro run(main())
Running that example will raise a TypeError with this message:
TypeError: expect an iterable of futures, not generatorIf we look at the first line in the body of as_completed(), we can see why this error is thrown for generators that yield awaitables:
def as_completed(fs, *, loop=None, timeout=None): if futures.isfuture(fs) or coroutines.iscoroutine(fs): raise TypeError(f"expect an iterable of futures, not {type(fs).__name__}") ...
Because generators are coroutines, and the first condition in as_completed() is True, and TypeError gets raised:
from asyncio import coroutines
# generators and generator expressions are coroutines assert coroutines.iscoroutine(example() for _ in range(10))Perhaps as_completed() can use inspect.isawaitable() instead, like so:
from inspect import isawaitable def as_completed(fs, *, loop=None, timeout=None): if futures.isfuture(fs) or isawaitable(fs): ...
I made a pull request with that change here[2].
[1] https://docs.python.org/3/library/asyncio-task.html#asyncio.as_completed
[2] #26228
3.12 allows generators so this is completed.
#103626 clarifies this in doc.
- added a commit that references this issue
on Apr 19, 2023 - added a commit that references this issue
on Apr 20, 2023
asyncio.as_completed()'s first parameter to be a generator yielding awaitables #26228Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs
asyncio.as_completedaccepts generators yielding tasks #103626