Repository navigation
PersistentTaskGroup API #90999
Description
| BPO | 46843 |
|---|---|
| Nosy | @gvanrossum, @asvetlov, @1st1, @Tinche, @achimnol |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2022-03-14.15:42:46.578>
created_at = <Date 2022-02-24.05:34:25.188>
labels = ['3.11', 'type-feature', 'expert-asyncio']
title = 'PersistentTaskGroup API'
updated_at = <Date 2022-03-14.15:42:46.578>
user = 'https://github.com/achimnol'bugs.python.org fields:
activity = <Date 2022-03-14.15:42:46.578>
actor = 'gvanrossum'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-14.15:42:46.578>
closer = 'gvanrossum'
components = ['asyncio']
creation = <Date 2022-02-24.05:34:25.188>
creator = 'achimnol'
dependencies = []
files = []
hgrepos = []
issue_num = 46843
keywords = []
message_count = 23.0
messages = ['413880', '413905', '413908', '413913', '413914', '413922', '413926', '413959', '413960', '413961', '413962', '413967', '414014', '414015', '414026', '414029', '414030', '414034', '414037', '414161', '414637', '415132', '415149']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'asvetlov', 'yselivanov', 'tinchester', 'achimnol']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46843'
versions = ['Python 3.11']
Activity
achimnol commentedon Feb 24, 2022
I'm now tracking the recent addition and discussion of TaskGroup and cancellation scopes. It's interesting! :)
I would like to suggest to have a different mode of operation in asyncio.TaskGroup, which I named "PersistentTaskGroup".
AFAIK, TaskGroup targets to replace asyncio.gather, ensuring completion or cancellation of all tasks within the context manager scope.
I believe that a "safe" asyncio application should consist of a nested tree of task groups, which allow us to explicitly state when tasks of different purposes and contexts terminate. For example, a task group for database transactions should be shutdown before a task group for HTTP handlers is shutdown.
To this end, in server applications with many sporadically spawned tasks throughout the whole process lifetime, there are different requirements for a task group that manages such task sets. The tasks should *not* be cancelled upon the unhandled exceptions of sibling tasks in the task group, while we need an explicit "fallback" exception handler for those (just like "return_exceptions=True" in asyncio.gather). The tasks belong to the task group but their references should not be kept forever to prevent memory leak (I'd suggest using weakref.WeakSet). When terminating the task group itself, the ongoing tasks should be cancelled. The cancellation process upon termination may happend in two phases: cancel request with initial timeout + additional limited waiting of cancellations. (This is what Guido has mentioned in the discussion in bpo-46771.)
An initial sketch of PersistentTaskGroup is on aiotools:
https://github.com/achimnol/aiotools/blob/main/src/aiotools/ptaskgroup.py
Currently has no two-phase cancellation because it would require Python 3.11 with asyncio.Task.uncancel().
As Andrew has left a comment (achimnol/aiotools#29 (comment)), I think it is the time to revisit the concrete API design and whether to include PersistentTaskGroup in the stdlib or not.
achimnol commentedon Feb 24, 2022
So I have more things in mind.
Basically PersistentTaskGroup resemble TaskGroup in that:
TaskGroup is intended to be used for a short-lived set of tasks, while PersistentTaskGroup is intended for a long-running set of tasks though individual tasks may be short-lived. Thus, adding globally accessible monitoring facility for plain TaskGroup would not be that useful. In contrast, it is super-useful to have a monitoring feature in PersistentTaskGroup!
In aiomonitor, we can enumerate the currently running asyncio tasks by reading asyncio.Task.all_tasks(). This has saved my life several times when debugging real-world server applications. I think we can go further by having asyncio.PersistentTaskGroup.all_task_groups() which works in the same way. If we make different modules and libraries to use different persistent task groups, then we could keep track of their task statistics separately.
achimnol commentedon Feb 24, 2022
I think people may ask "why in stdlib?".
My reasons are:
Of course, the name "PersistentTaskGroup" may look quite long, and I'm completely open with alternative suggestions. I also welcome suggestions on changes to its functional semantics based on your experience and knowledge.
achimnol commentedon Feb 24, 2022
Example use cases:
achimnol commentedon Feb 24, 2022
Some search results from cs.github.com with the input "asyncio task weakset", which may be replaced/simplified with PersistentTaskGroup:
https://github.com/mars-project/mars/blob/d1a14cc4a1cb96e40e1d81eef38113b0c9221a84/mars/lib/aio/_runners.py#L57
achimnol commentedon Feb 24, 2022
@yselivanov @asvetlov
I think this API suggestion would require more refining and discussion in depths, and probably it may be better to undergo the PEP writing and review process. Or I might need to have a separate discussion thread somewhere else (maybe discuss.python.org?).
Since I'm just a newbie in terms of Python core/stdlib development, could one of you guide me with what you think as the right way?
gvanrossum commentedon Feb 24, 2022
Could you just have a global task group that owns these long-running tasks? It could be embedded in a "toplevel" task that is created using asyncio.create_task() (which won't be deprecated). To shut down all long-running tasks at the end, just cancel that toplevel task.
achimnol commentedon Feb 25, 2022
@gvanrossum As you mentioned, the event loop currently plays the role of the top-level task group already, even without introducing yet another top-level task. For instance, asyncio.run() includes necessary shutdown procedures to cancel all belonging unfinished tasks and async generators.
However, I think we should provide an abstraction to organize the shutdown procedures in a *hierarchical* manner. For example, we could cancel all event handler tasks before cancelling all HTTP handler tasks upon a web server shutdown. This prevents any potential races between theses two different task sets. I think you could agree with the necessity of orderly release of underlying resources during shutdown in general. Currently asyncio.Task.all_tasks() is just a list created from WeakSet and we cannot guarantee which tasks will be cancelled first.
Yes, this can be done by manually writing codes to declare multiple WeakSets and a for-loop to cancel the contained tasks by enumerating over them, just like asyncio.run() does. With the new addition of TaskGroup and ExceptionGroup, this code does not require core changes of Python.
But I believe that this hierarchical persistent task group abstraction should be an essential part of the API and asyncio tutorials when writing server applications. asyncio.run() could be written by users, but I think the core devs have agreed with that it is an essential abstraction to be included in the stdlib. I'd like argue that hierarchical persistent task groups is the same case.
Though I named it "PersistentTaskGroup" because it looks similar to TaskGroup, but this name may be misleading. In PersistentTaskGroup, even when all tasks finish successfully, it does NOT terminate but keeps waiting for new tasks to be spawned. It terminates only when the outer task is cancelled or its shutdown() method is called. Note that belonging tasks may be either short-running or long-running, and this does not matter. The point is to shutdown any remaining tasks in an orderly manner. If you don't like the naming, please suggest alternatives.
achimnol commentedon Feb 25, 2022
This particular experience, https://github.com/lablup/backend.ai-agent/pull/331, has actually motivated me to suggest PersistentTaskGroup.
The program subscribes the event stream of Docker daemon using aiohttp as an asyncio task, and this should be kept running throughout the whole application lifetime. I first applied aiotools.TaskGroup to ensure shutdown of spawned event handler tasks, but I missed that it cancels all sibling tasks if one of the spawned tasks bubbles up an unhandled exception. This has caused silent termination of the subscriber task and led to a bug. We could debug this issue by inspecting aiomonitor and checking the existence of this task. After this issue, I began to think we need a proper abstraction of a long-running task group (NOTE: the task group is long-running. The lifetime of internal tasks does not matter).
Another case is that lablup/backend.ai#330.
One of our customer site has suffered from excessive CPU usage by our program. We could identify the issue by aiomonitor, and the root cause was the indefinite accumulation of peridoically created asyncio tasks to measure the disk usage of user directories, when there are too many files in them. Since the number of tasks have exceeded 10K, it was very difficult to group and distinguish individual asyncio tasks in aiomonitor. I thought that it would be nice if we could group such tasks into long-running groups and view task statistics separately.
2 remaining items
achimnol commentedon Feb 25, 2022
Anoter case:
lablup/backend.ai-manager#533
https://github.com/lablup/backend.ai-agent/pull/341
When shutting down the application, I'd like to explicitly cancel the shielded tasks, while keep them shielded before shutdown.
So I inserted
ptaskgroup.create_task()insideasyncio.shield(), so that the tasks are not cancelled upon the cancellation of their callers but they get cancelled when the server shuts down.This pattern is conveniently implemented with PersistentTaskGroup.
gvanrossum commentedon Feb 25, 2022
The implementation of asyncio.TaskGroup isn't all that complicated (and the new "cancel count" API helps). I recommend that you build one that satisfies your requirements yourself, or convince the authors of some other package like Quattro or aiotools to provide variations.
gvanrossum commentedon Feb 25, 2022
(FWIW I would close this issue but I'll wait to see if @asvetlov has something to add.)
Tinche commentedon Feb 25, 2022
The asyncio TaskGroup already uses a WeakSet for its children, so it's suitable for long-lived use.
As for errors in siblings aborting the TaskGroup, could you apply a wrapper to the scheduled coroutines to swallow and log any errors yourself?
Apart from the timeouts, that should get you a long way towards what you're describing.
achimnol commentedon Feb 25, 2022
Good to hear that TaskGroup already uses WeakSet.
When all tasks finish, PersistentTaskGroup should not finish and wait for future tasks, unless explicitly cancelled or shutdown. Could this be also configured with asyncio.TaskGroup?
I'm also ok with adding a simple option for such behavior to asyncio.TaskGroup instead of adding a whole new API/class.
achimnol commentedon Feb 25, 2022
Yes, this could be a simplest way to implement PersistentTaskGroup if TaskGroup supports "persistent" option to keep it running.
And just a question: I'm just curious about what happens if belonging tasks see the cancellation raised from their inner tasks. Sibling tasks should not be cancelled, and the outer task group should not be cancelled, unless the task group itself has requested cancellation. Could the new cancellation counter help this?
achimnol commentedon Feb 25, 2022
To achieve this by distinguishing cancellation from inner/outer tasks, TaskGroup._on_task_done() should be modified to skip setting _on_completed_fut because it should keep running. Swallowing exceptions in child tasks can be done without modifying TaskGroup, but this part requires changes of TaskGroup.
Another difference is the usage. Instead of relying on the async context manager interface, we would call "TaskGroup.shutdown()" separately from either directly in signal handlers or from cleanup methods of long-lived objects that have task groups as attributes.
And I also want to perform two-phase cancellation: instead of cancelling all tasks immediately as in current _abort(), have a configurable grace period until they have chances to complete and then cancel with additional timeout on cancellation itself to prevent hangs.
achimnol commentedon Feb 25, 2022
Short summary:
PersistentTaskGroup shares the followings from TaskGroup:
PersistentTaskGroup differs in that:
The API looks similar to TaskGroup with minor modification.
The semantics of a PersistentTaskGroup more resembles a nested event loop, in that it has its own set of tasks, it keeps running until closed, and it has its own fallback exception handler.
Note that current aiotools implementation lacks many details, such as two-phase cancellation. I'm going to implement more soon.
achimnol commentedon Feb 27, 2022
I have updated the PersistentTaskGroup implementation referring asyncio.TaskGroup and added more detailed test cases, which works with the latest Python 3.11 GitHub checkout.
https://github.com/achimnol/aiotools/pull/36/files
Please have a look at the class docstring.
There are two different usage: async context manager vs. attributes of long-lived objects.
One of the point is to "revive" asyncio.gather() with return_exceptions=True but let it handle/report exceptions immediately with customizable exception handler.
Currently two-phase shutdown is not implemented yet as I'm still thinking about how to adapt the current implementation.
achimnol commentedon Mar 7, 2022
I have released the new version of aiotools with rewritten TaskGroup and PersistentTaskGroup.
https://aiotools.readthedocs.io/en/latest/aiotools.taskgroup.html
aiotools.TaskGroup has small additions to asyncio.TaskGroup: a naming API and
current_taskgroupcontext variable.aiotools.PersistentTaskGroup is what I've described here, highlighting both async-with usage and long-lived object usage and
all_ptaskgroups()classmethod for the monitoring purpose except the two-phase graceful shutdown (future TODO).asvetlov commentedon Mar 14, 2022
I think we should close the PR now.
I'm open to the discussion resurrection in Python 3.12 or 3.13, when aiotools implementation will be battle-tested.
gvanrossum commentedon Mar 14, 2022
Okay.