Repository navigation
Add a StreamReaderBufferedProtocol #85451
Description
Activity
tontinton commented
on Jul 11, 2020 tontintonmannequinMannequinAuthorThis will greatly increase performance, from my internal tests it was about 150% on linux.
Using read_into instead of read will make it so we do not allocate a new buffer each time data is received.
tontinton commented
on Aug 29, 2020 tontintonmannequinMannequinAuthorbump
- changed the title
[-]Convert StreamReaderProtocol to a BufferedProtocol[/-][+]Add a StreamReaderBufferedProtocol[/+]on Aug 29, 2020 - changed the title
[-]Convert StreamReaderProtocol to a BufferedProtocol[/-][+]Add a StreamReaderBufferedProtocol[/+]on Aug 29, 2020 - linked a pull request that will close this issuegh-85451: Add StreamReaderBufferedProtocol #21446
on Jan 16, 2026 @webknjaz FTR, we do not link PRs to issues in general because of backports or if we need to do more things afterwards. We usually leave the responsibility of closing the issue to the core dev who merged the PR.
cc @kumaraditya303 for the validity and relevence of this FR.
- removed a link to a pull requestgh-85451: Add StreamReaderBufferedProtocol #21446
on Jan 18, 2026 @picnixz ah, I didn't realize, reverted that. Thanks! I was going through a bunch of issues and noticed incorrect links in the OP, I corrected that by commenting one out (left a note in the OP to see if you click Edit) and assumed this was a part of the breakage. I'll stop linking PRs, then (there may be one or two others that I linked left, though).
I found a severe Linux allocator/pathological-latency case that makes the
BufferedProtocol/recv_into()direction here more than a steady-state micro-optimization. It is reproducible without any third-party library.Environment where it reproduces reliably enough to characterize:
- CPython 3.14.7
- Linux aarch64, Raspberry Pi 5 (Cortex-A76)
- glibc 2.43
- selector event loop
The selector transport still has
max_size = 256 * 1024and the ordinaryProtocolpath callssock.recv(self.max_size). With small socket arrivals,socket.recv(262144)can enter a process-layout-dependent allocator regime where each small read effectively causesmmap -> recvfrom -> mremap -> munmap.A minimal raw-socket test is enough to reproduce it. The important shape is one small payload outstanding at a time and a fresh client process per repetition:
# server process import socket ls = socket.socket() ls.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) ls.bind(("127.0.0.1", 9999)) ls.listen() while True: c, _ = ls.accept() with c: c.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) while True: c.sendall(b"x" * 256) if not c.recv(1): break
# run this client as a fresh process repeatedly import resource, socket, time s = socket.create_connection(("127.0.0.1", 9999)) s.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) for _ in range(2000): assert len(s.recv(262144)) == 256 s.sendall(b"A") a = resource.getrusage(resource.RUSAGE_SELF) t0 = time.perf_counter() for _ in range(20000): assert len(s.recv(262144)) == 256 s.sendall(b"A") dt = time.perf_counter() - t0 b = resource.getrusage(resource.RUSAGE_SELF) print("us/read", dt * 1e6 / 20000) print("minor_faults/read", (b.ru_minflt - a.ru_minflt) / 20000) print("stime_us/read", (b.ru_stime - a.ru_stime) * 1e6 / 20000)
On the same host, 8 fresh-process repetitions of raw
socket.recv(262144)split into 5 fast processes at ~12.45-12.57 us/read with 0 minor faults/read and 3 slow processes at ~14.87-15.05 us/read with exactly ~2.0 minor faults/read. Major faults were always 0.Controls:
socket.recv(65536): 8/8 at 0 minor faults/read, ~12.49 us/read median.- reusable
bytearray(262144)+socket.recv_into(): 8/8 at 0 minor faults/read and ~12.27 us/read median. - asyncio
StreamReader/ selector with the defaultmax_size=262144: 2/16 fresh processes at exactly ~2.0 minor faults/read, ~35.5-36.0 us/read vs ~22.2-23.0 us/read in the fast regime. - the same asyncio path with
transport.max_size=65536: 16/16 at 0 minor faults/read, ~22.36 us/read median. setarch aarch64 -Rwith the 256 KiB read: 16/16 at 0 minor faults/read.GLIBC_TUNABLES=glibc.malloc.mmap_threshold=1048576with the 256 KiB read: 16/16 at 0 minor faults/read.
A slow raw
recv(262144)run understrace -cfor 6000 measured reads produced:6000 mremap 6079 mmap 6049 munmap 6000 recvfromThe
recv_into()and 64 KiB variants had no per-readmremap; only normal startup mappings remained.This looks closely related to #121313:
recv(n)has to allocate abytesfor the requested size before it knows how much the kernel will return, then shrink it. At 256 KiB this crosses glibc's mmap territory, and depending on process/allocator layout the allocate-short-read-resize-free cycle can become dramatically more expensive. The process-layout dependence explains why it may be invisible on many machines; I could not reproduce the bimodality on an x86-64 KVM host under the same application-level workload.The full controlled run and raw evidence are public here: https://github.com/yoch/mqttium/actions/runs/34188738358 . Although that repository is MQTT-related, this particular run contains a standalone socket reproducer and does not import MQTTium or use an MQTT broker.
So there seem to be two possible upstream remedies worth considering:
- the structural solution proposed by this issue/PR: use a reusable buffer +
recv_into()for streams; or - independently, reconsider the selector transport's 256 KiB default read size for ordinary
Protocols, since a 64 KiB cap completely eliminated this allocator regime in the experiment without requiring a persistent per-stream 256 KiB buffer.
I can provide a smaller CPython-repo benchmark/test if useful.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: