Repository navigation
fix(cache): count memory that row-group snapshots hold - #104
Merged
Merged
Conversation
rohan-hotdata
requested review from
eddietejeda
and removed request for
a team
November 18, 2026 06:38
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
📊 Benchmark ComparisonCurrent:
Compared Liquid vs DataFusionDefault on the same runner |
A row group's snapshot map kept arrays alive after the cache evicted them, so the memory counter dropped while the memory stayed. - The reader drops each batch's snapshots once it has emitted or skipped the batch, instead of at the end of the row group. - Core records the snapshots that prefetch hands out. A memory entry that leaves the index while one of its snapshots is alive stays charged until the snapshot is dropped. Usage reads and refused reservations release charges whose snapshots are gone. - The first row group's prefetch waits for headroom like later row groups. A prefetch stops when the cache refuses an insert, rather than keeping an uncounted copy, and snapshots a disk entry only if hydration puts it back in memory. Closes #79
rohan-hotdata
force-pushed
the
fix/snapshot-accounting
branch
from
November 18, 2026 12:05
a389599 to
466feec
Compare
anoop-narang
added a commit
that referenced
this pull request
Nov 18, 2026
Resolve against the snapshot accounting from #104: sweep_held releases through the saturating release, and try_reserve_memory sweeps held snapshots and retries once when the reservation is refused. Claude-Session: https://claude.ai/code/session_01LbJrgYNLF7FxwMy5VSHXar
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes the cache's memory counter include arrays that a scan's row-group snapshots keep alive. Before, the cache sometimes evicted an entry that a snapshot still held. Then the counter dropped and the memory stayed.
Closes #79
How
Three changes, one per way a snapshot escaped the counter.
A. The reader frees each batch's snapshots after the batch.
RowGroupSnapshots::release_batchdrops a batch's entries and selection.LiquidCacheReader::next_batchcalls it once the batch is emitted or skipped. Before, a row group's snapshots lived until the stream ended. On a full cache, that parked the whole row group (liquid_cache_reader.rs,cache/mod.rs).B. Core keeps a snapshotted entry charged after it leaves the index.
LiquidCache::prefetchregisters each snapshot it hands out (BudgetAccounting::register_snapshot). Every path that takes a memory entry out of the index (try_insert's displaced slot,drop_memory_victim,drop_memory_entry) callshold_if_snapshotted. If a live snapshot holds the same array, its bytes stay charged on a held list.memory_usage_bytes()and a refusedtry_reserve_memorysweep that list and release entries whose snapshots are gone (budget.rs,core.rs).Only snapshots count. A first version held any entry whose array had another reference. That charged arrays the inserter keeps, and the reader also passes inserted arrays downstream to DataFusion.
or_fast_path_evaluates_on_liquidcaught it: the cache dropped entries that the test still needed.C. Prefetch respects the budget.
has_headroom_for_front).materialize_prefetch_batchstops onCacheFull. Before, it kept a snapshot of the refused array, which nothing counted.prefetch, so they are registered.prefetchnow checks whether hydration put it back in memory. If it did,prefetchsnapshots it. If not,prefetchreports it absent. Hydration decides as before. Withmax_disk_bytes(0), as RuntimeDB runs, this path never runs.Verification
audit/memory-testsas assertions, plus memory-only variants. 5 fail onmainand pass here. They assert that the bytes the index holds plus the bytes only snapshots hold, each array counted once, stay within the counter.concurrent_prefetch_snapshots_stay_within_the_counterand_memory_onlya_reader_on_a_full_cache_snapshots_at_most_the_batch_it_readsevicted_snapshotted_entries_stay_chargedfirst_row_group_prefetch_waits_for_headroomcargo test --workspace, the shuttle tests,cargo clippy --workspace --exclude dev-tools --all-targets --all-features -D warningsandcargo fmt --checkall pass. The test inventory againstmainonly adds tests (9 default, 4 shuttle).Notes
main. The concurrent case is real: onmain, 4 prefetches hold 1.62x the budget (1.38x with disk 0).main, the counter missed 30.1 to 31.1 MiB of heap. On this branch it missed 28.9 to 30.2 MiB. That is about 1 MiB apart, inside the noise. Prefetch ran in both. About 5 MiB of the gap stays after the queries finish (likely liquid: entries are charged without their per-node overhead #76), and the rest is transient while queries run (likely Memory budget excludes hydrate/squeeze transients: tier reports at limit while the process holds 3x that #43, see fix(cache): count and bound bytes held outside the memory index #44). That measurement is not in the PR. It was a one-off allocator test.PlanOptionsandprepare_test_file), and the resolution keeps both sides.headroom_gate_sizes_row_groups_by_decoded_bytesassumed that the first row group is always prefetched. With the first-row-group gate, it is not. That made the test pass on a thin margin: the cache was empty, so it had 583,320 bytes free against 557,180 decoded bytes. The test now gives the cache exactly the intended headroom: 4x encoded, 26,140 bytes. It also asserts that the cache is empty after the first row group. Its main assertion is unchanged. With the old encoded-size estimate swapped back in, it still fails with its own message.try_insertand the drop paths incore.rsandbudget.rs. It will conflict textually, but not in design: fix(cache): keep the memory budget matching the index #100 fixes how the counter settles.getis not charged after eviction (only prefetch snapshots are). A snapshot taken in the window between an entry's removal andhold_if_snapshottedis not held. That window is a single batch, and the reader frees the batch afterwards.