Repository navigation
fix(cache): keep the memory budget matching the index - #100
Conversation
A release larger than the usage wrapped the counter to near usize::MAX, after which the tier refused every reservation, and a reservation whose sum overflowed panicked in debug and wrapped in release. Releases now stop at zero and reservations that overflow are refused. Closes #72
try_insert moved the memory budget by the size of the slot it read, then installed over whatever held the key by then. Two inserts of one key, or an eviction racing a re-insert, left the budget over- or under-counting the index for good. The insert now corrects the budget by the size of the slot it actually displaced. Closes #67
insert_inner takes up to 8 victims off the policy's queues, and only a victim whose eviction installs its new form is filed again. A victim whose eviction failed, or never ran because the insert was dropped, stayed in the index and charged with no queue offering it. With no disk tier this is every refused insert: a Liquid victim has nowhere to go, so evict_victims fails with CacheFull. Victims are now held in a lease that files each unfinished one back with the policy. Closes #66
evict_victims failed the whole insert as soon as any victim failed, even when other victims had already freed enough memory. With no disk tier a batch of victims mixes Arrow entries, which transcode, and Liquid ones, which cannot move, so an insert that fit was refused with CacheFull. It now fails only when no victim was evicted, and the insert retries. Closes #69
insert_inner always took 8 memory victims, whatever the insert needed. One insert into a full tier of Arrow entries transcoded 8 of them, each costing an encode and leaving data slower to read. Victims are now taken one at a time until what they hold covers what the budget lacks, up to 8; an insert that still lacks room comes back for more. Two datafusion tests relied on the 8-victim batch to transcode both their columns; their dummy column is now wide enough to need both. Refs #70
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
📊 Benchmark ComparisonCurrent:
Compared Liquid vs DataFusionDefault on the same runner |
Rust 1.100 deprecates AtomicUsize::fetch_update in favour of try_update, and CI's clippy denies warnings. try_update is missing from older toolchains and from shuttle's atomics, so the update loop is written out with compare_exchange_weak.
Brings in #103, which fixes the memory-only freeze and settles try_insert's charge against the slot actually displaced. - try_insert: #103's growth-only reservation and settle replace ours (#67); settle_memory_release goes, and its test becomes one for #103's charge_memory. - budget.rs: our reserve/release keep the overflow and underflow guards (#72); charge_memory saturates through update() rather than fetch_add. - insert_inner: #103's guard for an entry larger than the memory tier, then find_memory_victims_for (#70). - Two #66 tests assumed a full memory-only cache refuses inserts; they now check it admits and keeps the rest evictable. - Snapshots provide_schema2 and url_selection_and_ordering: victims by need transcode fewer Arrow entries, and shift one more entry through the latter's hydrate rotation.
There was a problem hiding this comment.
The merge of origin/main is consistent with this branch. In try_insert, the reserve-then-settle split charges or releases the gap between the slot read and the slot displaced, through charge_memory and the saturating release. evict_victim_inner now always returns Ok, so evict_victims fails only for an empty victim list. Dropped victims leave the VictimLease through evicted, and a newer entry is filed again by drop_memory_victim. CI was still running when this review started, so no test result is confirmed here.
Resolve against the snapshot accounting from #104: sweep_held releases through the saturating release, and try_reserve_memory sweeps held snapshots and retries once when the reservation is refused. Claude-Session: https://claude.ai/code/session_01LbJrgYNLF7FxwMy5VSHXar
…rowth - try_insert charges the part of an entry it did not reserve before the entry enters the index, and releases the slot it displaced after. A removal that follows the insert at once never finds the entry uncharged, so a saturating release cannot clip and leave a permanent over-count. - An insert that replaces a key sizes its eviction by the growth over the slot it replaces, which is what try_insert reserves, not the whole entry. - evict_victims and evict_victim_inner no longer return a Result: a victim the disk cannot take is dropped, so eviction cannot fail, and the partial failure handling was unreachable. VictimLease still re-offers victims of an eviction dropped while it waits on a disk write. - A budget release larger than the usage is counted in the new budget_underflows runtime stat and logged once. - Tests: try_reserve_memory sweeps dropped snapshots and retries; a replacement evicts only for its growth. Claude-Session: https://claude.ai/code/session_01LbJrgYNLF7FxwMy5VSHXar
Keeps the memory budget counting what the index holds, and keeps eviction from taking more than an insert needs.
budget_underflowsruntime stat and logged once; a reservation that would overflowusizeis refused. (cache: budget counters wrap below zero #72)VictimLease). Only an eviction waiting on a disk write can be dropped part-way, so this matters with a disk tier. (cache: failed or cancelled eviction strands its victims #66)evict_victims/evict_victim_innerno longer return aResult: since fix(cache): keep admitting once a memory-only cache is full #103 a victim the disk cannot take is dropped, so eviction cannot fail.Settling an insert against the slot it displaced (#67) is already on
mainthrough #103 and is only tested here. #69 (one failed victim refusing the insert) cannot happen since #103.Closes #72
Closes #66
Refs #70
Refs #67
Refs #69
https://claude.ai/code/session_01LbJrgYNLF7FxwMy5VSHXar