You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
liquid: decode copies misaligned buffers and keeps the block #77
Found by a memory-accounting audit of 8b3c813. The tests are on branch audit/memory-tests and fail on main today; check out that branch to run them. File and line links point at that commit.
What goes wrong.
When a buffer lands at a misaligned offset in the serialized block, decoding makes an aligned copy. Vortex's ensure_aligned (vortex-array/src/serde.rs:692,703) is the inferred source.
The other buffers still pin the whole block, so the copied bytes are held twice, and nbytes() counts them once.
Whether an entry pays depends on its byte layout: the same Binary data with nulls copied nothing.
Evidence.bytes in separate allocations == 0 failed for 21 of 41 fixtures, 8 of them material:
Fixture
Bytes copied
Charged → real
Binary random 16 B (varbinview.views)
131,072
262,184 → 394,588 (1.51x)
Timestamp ramp (bitpacked.packed)
20,520
20,587 → 48,123
FSST strings (bitpacked.packed)
19.5–21 KB each
This also resolves the earlier timestamp discrepancy (27.5 KB vs 2.5 KB uncharged): only one of the two fixtures hit the copy.
cargo +1.96.0 test -p liquid-cache --lib audit_decode_copies_no_buffer_out_of_the_serialized_block -- --nocapture
Flags: P probabilistic (a pass after a fix only means the race wasn't hit), F interleaving forced by a test hook, L Linux only, D needs Docker, M measurement only, C control (passes by design).
Activity
fix(liquid): keep decoded buffers in the block