Repository navigation
Reduce asyncio heapq overhead #122881
Description
Feature or enhancement
Proposal:
The heapq calls in base_events.py represents quite a bit of asyncio scheduling overhead because they have to run __lt__ quite often in TimerHandle
Line 128 in 0fd97e4
cpython/Lib/asyncio/base_events.py
Line 1968 in 0fd97e4
cpython/Lib/asyncio/base_events.py
Line 815 in 0fd97e4
cpython/Lib/asyncio/base_events.py
Line 1975 in 0fd97e4
Avoiding running __lt__ can speed up processing call_ats by ~10%
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
Wrapping TimerHandle in a tuple starting with when avoids the __lt__ call. Thank you to whoever wrote the heapq docs for help getting there https://docs.python.org/3/library/heapq.html#basic-examples
6f80b4c
Example benchmark
from asyncio import TimerHandle
import heapq
import timeit
def callback():
"""This is the callback function that will be called when the timer expires."""
class MockLoop:
def get_debug(self):
return False
loop = MockLoop()
def heap_tuple():
scheduled = []
when = 1
for _ in range(100):
when += 1
handle = TimerHandle(when, callback, (), loop)
heapq.heappush(scheduled, (when, handle))
while scheduled:
when, handle = heapq.heappop(scheduled)
def heap_handle():
scheduled = []
when = 1
for _ in range(100):
when += 1
handle = TimerHandle(when, callback, (), loop)
heapq.heappush(scheduled, handle)
while scheduled:
handle = heapq.heappop(scheduled)
print("wrap when, TimerHandle in tuple", timeit.timeit(heap_tuple))
print("bare TimerHandle", timeit.timeit(heap_handle))% python3 bench/timer_handle_heap.py
wrap when, TimerHandle in tuple 34.082984749999014
bare TimerHandle 49.678519583001616
Activity
bdraco commentedon Aug 10, 2024
call_at benchmark with the change in #122882
rhettinger commentedon Aug 10, 2024
Wow, this was an excellent problem analysis.
Your recommended solution seems reasonable to me.
vstinner commentedon Sep 23, 2024
I close the issue since the related PR was closed.