You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Unbound stream upload buffers use about twice the documented memory #666
For uploads from an unbound stream, each buffer is a bytearray, and UnboundSourceBytes wraps it in IOWrapper, a BytesIO subclass that copies the data. Each in-flight chunk is therefore held twice, so the memory in use is about double what buffer_size_bytes and queue_size suggest.
Where
b2sdk/_internal/transfer/emerge/unbound_write_intent.py:51 (IOWrapper.__init__), :78-86 (UnboundSourceBytes.__init__), :188-195 (_trim_to_leftovers copies the buffer).
Why it matters
Impact: Availability (memory use above the documented amount), silent. Severity: Low, because it happens on the normal unbound-stream path, but no concrete failure is shown.
How to check
Upload a stream with a large buffer_size_bytes and watch peak memory (for example with tracemalloc). Found by reading the code, not reproduced.
Version
Seen at v2.13.4. Present on the default branch: present.
Activity