Repository navigation
fix(cache): keep admitting once a memory-only cache is full - #103
Merged
Merged
Conversation
With no disk tier, eviction spilled a Liquid victim to disk, the write failed with CacheFull, and the insert was refused without freeing anything. A full cache then kept what it cached first and refused every new entry. Eviction now drops a memory victim the disk cannot take, and drops a Liquid victim before serialising it when the disk tier is zero. The drop removes the slot only if it still holds the entry the eviction picked, so a newer write of the key survives. An entry larger than the whole memory tier no longer evicts anything before it is refused. Dropping victims means a batch's earlier columns can leave the cache while its later ones are inserted. The reader evaluated predicates from the cache after filling it, and failed with "predicate unavailable after materialization". It now falls back to the batch it just read.
try_insert charged the memory budget against the slot it read, then installed over whatever slot was there by then. A concurrent drop or replacement of the key in between left the budget off by that slot's size, and enough of that can wrap the counter and refuse every insert. The charge is now settled against the slot actually displaced. A drop that removed a key and then told the policy could also remove a newer entry of the key from the policy, leaving it unevictable. The drop now files the key again if the index holds it.
Releasing a shrink before the insert could under-read the budget while another write of the key raced it. Only growth is now reserved up front, and the rest is settled once the insert knows what it displaced, so the budget can over-read for a moment but never under-read. Also test the re-file of a key written again while its victim drops, and note that a policy's notify_insert must be an upsert.
anoop-narang
requested review from
rohan-hotdata
and removed request for
a team
November 17, 2026 20:32
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
📊 Benchmark ComparisonCurrent:
Compared Liquid vs DataFusionDefault on the same runner |
zfarrell
added a commit
that referenced
this pull request
Nov 18, 2026
Brings in #103, which fixes the memory-only freeze and settles try_insert's charge against the slot actually displaced. - try_insert: #103's growth-only reservation and settle replace ours (#67); settle_memory_release goes, and its test becomes one for #103's charge_memory. - budget.rs: our reserve/release keep the overflow and underflow guards (#72); charge_memory saturates through update() rather than fetch_add. - insert_inner: #103's guard for an entry larger than the memory tier, then find_memory_victims_for (#70). - Two #66 tests assumed a full memory-only cache refuses inserts; they now check it admits and keeps the rest evictable. - Snapshots provide_schema2 and url_selection_and_ordering: victims by need transcode fewer Arrow entries, and shift one more entry through the latter's hydrate rotation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A cache built with
with_max_disk_bytes(0)(memory only) stopped admitting new data once its memory filled: it kept whatever it cached first and refused every later insert. This change makes a full memory-only cache drop old entries to make room, keeps filtered queries correct while it does, and fixes two accounting races that the new drop path made easier to hit.Problem
Eviction moves an Arrow entry to Liquid in memory, and a Liquid entry to disk. With no disk tier, the disk write of a Liquid victim failed with
CacheFull,evict_victimsreturned the error, andinsert_innergave up without freeing anything. Once memory held only Liquid entries, every insert was refused.Reproduced with
LiquidCacheLocalBuilder, an 8 MiB memory budget,max_disk_bytes(0), default policies, and an admission gate of 4/1/1/strict: after six tables fill the cache, a seventh table queried three times caches 1 of its 25 batches, with 72 refused inserts. With a disk tier the same table caches 25 of 25.Change
Eviction (
src/core/src/cache/core.rs)drop_memory_victim). With no disk tier, a Liquid victim is dropped before the eviction policy serialises it for a write that cannot happen.Accounting (
try_insert,BudgetAccounting)try_insertcharged the budget against the slot it read, then installed over whatever slot was there by then. A concurrent drop or replacement of the key in between left the budget off by that slot's size; enough of that can wrap the counter and refuse every insert. Only growth over the slot read is now reserved up front, and the rest is settled against the slot actually displaced. The budget can over-read for a moment, never under-read.try_reserve_memoryuses a saturating add.Reader (
src/datafusion/src/reader/runtime/liquid_cache_reader.rs)predicate unavailable after materialization(seen on Boolean and timestamp-with-time-zone filter columns, which Liquid does not encode). It still tries the cache first, and now falls back to evaluating the predicates on the record batch it just read.Behaviour changes worth knowing
insert_wont_fit_cache: the oversized entry no longer evicts entry 0 before going to disk.test_provide_schema2: an oversized batch is transcoded instead of first spilling two other entries to disk; results are unchanged.Tests
src/datafusion-local/tests/memory_only_freeze.rs: the reproduction above, and its disk-tier control.src/datafusion-local/tests/memory_only_workloads.rs, every result checked against DataFusion with no cache, and after each phase the charged memory checked against the bytes the index holds:main(nothing cached after the first pass); the filter test with the cache fix but not the reader fix (predicate unavailable after materialization); the accounting stress test without the settle (5 of 5 runs off by up to 13 KB); the re-file test without the re-file.morselizer.rsasserted the old behaviour (a full cache keeps its first batches and refuses the last); they now assert that the last batch is cached.cargo fmt --check,cargo clippy --workspace --exclude dev-tools --all-targets --all-features -- -D warnings, andcargo test --workspace --exclude dev-toolspass on Rust 1.96.