Repository navigation
asynciomodule.c performance degradation in free-threading under heavy load #144337
Description
Bug report
Bug description:
We have asyncio Python servers we are trying to run under free-threading that create event loops. Under, say, 800 QPS we noticed performance regressions relative to the GIL build and, upon tracing, most of the regression is in the cancel_all_tasks call when it calls the C implementation to get all tasks for the loop in question. My LLM suggested that the performance bottleneck is in the _PyEval_StopTheWorld call required. It suggested an alternative implementation using per-thread task buckets to avoid needing to stop the world.
It created the following repro script to show the performance issue. With its suggested changes it creates 10X loops and increases churn rate 83X under free-threading (changes would be ifdef-guarded to avoid a 10X regression in loop churn rate under standard build).
I still need to validate the changes it proposed but, without doing something, we would have to avoid asyncio + free-threading because of this.
import asyncio
import threading
import time
import argparse
import sys
# Configuration
NUM_CHURN_THREADS = 20 # Number of threads creating/destroying loops
RUN_DURATION = 10 # Seconds to run
TARGET_LOOPS_PER_SEC = 800 # Total target loops per second across all threads
stop_event = threading.Event()
def churn_worker(worker_id):
"""Continuously creates and destroys event loops."""
count = 0
while not stop_event.is_set():
# asyncio.run creates a new event loop, runs the coroutine, and closes it.
asyncio.run(asyncio.sleep(0))
count += 1
# Optional: slight delay to throttle if needed, but we want max contention for this demo
# time.sleep(0.001)
return count
async def monitor_coro():
"""Calls all_tasks continuously."""
count = 0
start_time = time.time()
while not stop_event.is_set():
# This triggers the scan of all tasks in all threads
tasks = asyncio.all_tasks()
count += 1
# Yield to allow other things to happen on this loop, though we mostly care about the scan cost
await asyncio.sleep(0)
duration = time.time() - start_time
return count, duration
def monitor_worker(results_list):
"""Runs a persistent loop that calls all_tasks."""
try:
count, duration = asyncio.run(monitor_coro())
results_list.append((count, duration))
except Exception as e:
print(f"Monitor failed: {e}")
def main():
parser = argparse.ArgumentParser(description="Asyncio Performance Demo")
parser.add_argument("--threads", type=int, default=NUM_CHURN_THREADS, help="Number of churn threads")
parser.add_argument("--duration", type=int, default=RUN_DURATION, help="Duration in seconds")
args = parser.parse_args()
print(f"Starting generic asyncio benchmark on {sys.version}...")
print(f"Configuration: {args.threads} churn threads, {args.duration}s duration.")
threads = []
# Start churn threads
# We use a ThreadPoolExecutor or just raw threads. Raw threads are fine.
# To measure iterations, we can use a mutable list or class
churn_counts = [0] * args.threads
def wrapped_churn(idx):
churn_counts[idx] = churn_worker(idx)
for i in range(args.threads):
t = threading.Thread(target=wrapped_churn, args=(i,))
t.start()
threads.append(t)
# Start monitor thread
monitor_results = []
monitor_thread = threading.Thread(target=monitor_worker, args=(monitor_results,))
monitor_thread.start()
threads.append(monitor_thread)
# Run for specified duration
try:
time.sleep(args.duration)
except KeyboardInterrupt:
pass
finally:
stop_event.set()
# Join all
for t in threads:
t.join()
# Aggregate results
total_loops = sum(churn_counts)
loops_per_sec = total_loops / args.duration
monitor_calls, mon_duration =0, 1
if monitor_results:
monitor_calls, mon_duration = monitor_results[0]
all_tasks_per_sec = monitor_calls / mon_duration if mon_duration > 0 else 0
print("-" * 40)
print(f"Results:")
print(f" Total Event Loops Created: {total_loops}")
print(f" Loop Churn Rate: {loops_per_sec:.2f} loops/sec")
print(f" all_tasks() Calls: {monitor_calls}")
print(f" all_tasks() Rate: {all_tasks_per_sec:.2f} calls/sec")
print("-" * 40)
print("Interpretation:")
print(" Higher 'Loop Churn Rate' indicates less blocking during thread/task destruction.")
print(" Higher 'all_tasks() Rate' indicates faster scanning of tasks.")
print(" On unoptimized Python (with StopTheWorld), these numbers should be significantly lower")
print(" concurrently due to the global pause.")
if __name__ == "__main__":
main()CPython versions tested on:
3.14
Operating systems tested on:
Linux
Activity
Muawiya-contact commentedon Jan 30, 2026
Hi @sergey-miryanov @StanFromIreland, I’m interested in working on this issue. Could you please assign me so I can start investigating the performance regression?
StanFromIreland commentedon Jan 30, 2026
We generally don't assign issues here.
Muawiya-contact commentedon Jan 31, 2026
@StanFromIreland Sorry about that — understood. I won’t ask for the assignment again. I’ll just follow along and contribute via a PR or discussion if I have something useful. Thanks for the clarification.
picnixz commentedon Jan 31, 2026
Also please always read the devguide. I assume you are using an LLM to answer but we do not accept PRs that are generated by AI and users abusing them are banned from contributing.
kumaraditya303 commentedon Feb 1, 2026
Why are you creating and destroying event loops? A server would typically create one loop per thread and handles requests, the current reproduce feels like an artificial AI generated script to me which does not represents a real use-case.
jgrave3 commentedon Feb 2, 2026
To answer the question as to why it is creating and destroying event loops? That is actually in the existing code others were writing that I am load testing under 3.14t. The problem is that our server code (C++ under the hood) is spinning up a new thread per request and thus an event loop per. I agree it would be more reasonable is to have N event loops corresponding to the number of cores and distribute work among them. That will be a bit of an adventure for us to unwind and implement and, even then, the shared mutables across N event loops are still a synchronization pain. More likely we will just ban either asyncio or free-threading for serving.
As I said above, the repro script I included was LLM-generated, but that is to provide something vanilla Python to illustrate the current situation and to provide a benchmark. I left out the LLM-generated runners.py version which bumped these metrics up 10X and 83X. So if we want to be defensive against overuse of asyncio.run, that is also possible.
colesbury commentedon Feb 2, 2026
Thanks for the explanation. I am still a bit confused about which APIs are actually being called. For example, the description mentions
cancel_all_tasks(), the script callsasyncio.all_tasks()(that is, across all event loops), and we have also discussed creating and destroying event loops.Because of this, I am not sure that the LLM-generated repro demonstrates the same problem you are encountering, rather than a related scaling bottleneck specific to
asyncio.all_tasks().For context, the asyncio free-threading design prioritizes making common use cases fast, even if that comes at the expense of less common ones. For example, operations within a single event loop generally do not require locking, while operations that span all event loops may require a stop-the-world pause. This is in contrast to an approach that would require locking everywhere, which would be more balanced but slower in the common case. Because of this tradeoff, it is important for us to understand the actual use cases involved, rather than relying on a script that only demonstrates that something is slow.
jgrave3 commentedon Feb 3, 2026
kumaraditya303 commentedon Feb 3, 2026
The fast-path is only present in 3.15, in 3.14 the stop-the-world pause happens in all cases.
Can you test your code on 3.15 main branch?
jgrave3 commentedon Feb 4, 2026
colesbury commentedon Feb 4, 2026
@kumaraditya303, should we backport #140542 to 3.14?
kumaraditya303 commentedon Feb 5, 2026
Yes I think we should backport it as the change is simple.
kumaraditya303 commentedon Feb 5, 2026
Backport PR #144494
kumaraditya303 commentedon Feb 15, 2026
Thanks for the report, the change is backported so I am closing this issue. If you find more performance issues in asyncio feel free to file a new issue.