Skip to content

fix(cache): count memory that row-group snapshots hold - #104

Merged
rohan-hotdata merged 1 commit into
mainfrom
fix/snapshot-accounting
Nov 18, 2026
Merged

rohan-hotdata merged 1 commit into
mainfrom
fix/snapshot-accounting

Conversation

@rohan-hotdata

@rohan-hotdata rohan-hotdata commented Nov 18, 2026 •

Copy link
Copy Markdown

This PR makes the cache's memory counter include arrays that a scan's row-group snapshots keep alive. Before, the cache sometimes evicted an entry that a snapshot still held. Then the counter dropped and the memory stayed.

Closes #79

How

Three changes, one per way a snapshot escaped the counter.

A. The reader frees each batch's snapshots after the batch. RowGroupSnapshots::release_batch drops a batch's entries and selection. LiquidCacheReader::next_batch calls it once the batch is emitted or skipped. Before, a row group's snapshots lived until the stream ended. On a full cache, that parked the whole row group (liquid_cache_reader.rs, cache/mod.rs).

B. Core keeps a snapshotted entry charged after it leaves the index. LiquidCache::prefetch registers each snapshot it hands out (BudgetAccounting::register_snapshot). Every path that takes a memory entry out of the index (try_insert's displaced slot, drop_memory_victim, drop_memory_entry) calls hold_if_snapshotted. If a live snapshot holds the same array, its bytes stay charged on a held list. memory_usage_bytes() and a refused try_reserve_memory sweep that list and release entries whose snapshots are gone (budget.rs, core.rs).

Only snapshots count. A first version held any entry whose array had another reference. That charged arrays the inserter keeps, and the reader also passes inserted arrays downstream to DataFusion. or_fast_path_evaluates_on_liquid caught it: the cache dropped entries that the test still needed.

C. Prefetch respects the budget.

  • The first row group now passes the same headroom gate as later ones (has_headroom_for_front).
  • materialize_prefetch_batch stops on CacheFull. Before, it kept a snapshot of the refused array, which nothing counted.
  • It takes snapshots through prefetch, so they are registered.
  • For a disk entry, prefetch now checks whether hydration put it back in memory. If it did, prefetch snapshots it. If not, prefetch reports it absent. Hydration decides as before. With max_disk_bytes(0), as RuntimeDB runs, this path never runs.

Verification

  • Ported the audit tests for reader: row-group snapshots pin batches outside the budget #79 from audit/memory-tests as assertions, plus memory-only variants. 5 fail on main and pass here. They assert that the bytes the index holds plus the bytes only snapshots hold, each array counted once, stay within the counter.
    • concurrent_prefetch_snapshots_stay_within_the_counter and _memory_only
    • a_reader_on_a_full_cache_snapshots_at_most_the_batch_it_reads
    • evicted_snapshotted_entries_stay_charged
    • first_row_group_prefetch_waits_for_headroom
  • 4 new core tests for B, which also run under shuttle:
    • A snapshot keeps an evicted entry charged.
    • A transcoded entry keeps its snapshotted Arrow charged.
    • An array the inserter keeps is not charged.
    • Held snapshots block admission until they are dropped.
  • cargo test --workspace, the shuttle tests, cargo clippy --workspace --exclude dev-tools --all-targets --all-features -D warnings and cargo fmt --check all pass. The test inventory against main only adds tests (9 default, 4 shuttle).

Notes

@rohan-hotdata
rohan-hotdata requested a review from a team as a code owner November 18, 2026 06:38
@rohan-hotdata
rohan-hotdata requested review from eddietejeda and removed request for a team November 18, 2026 06:38
Comment thread src/core/src/cache/core.rs Resolved
claude[bot]
claude Bot previously approved these changes Nov 18, 2026
@codecov

codecov Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.15195% with 9 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/datafusion/src/cache/mod.rs 94.59% 4 Missing ⚠️
src/datafusion/src/reader/plantime/morselizer.rs 97.95% 2 Missing and 2 partials ⚠️
src/core/src/cache/core.rs 99.25% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

📊 Benchmark Comparison

Current: 37818072 (Liquid) vs Baseline: 37818072 (DataFusionDefault)

Query Cold Time Δ Warm Time Δ CPU Time Δ
Q1 2.0ms (3.0ms) -33.3% 0.000ms (0.000ms) +0.0% 0.000ms (0.000ms) +0.0%
Q2 12.0ms (7.0ms) +71.4% 4.0ms (5.0ms) -20.0% 8.0ms (7.0ms) +14.3%
Q3 16.0ms (14.0ms) +14.3% 6.0ms (13.0ms) -53.8% 4.0ms (24.0ms) -83.3%
Q4 16.0ms (11.0ms) +45.5% 4.0ms (9.0ms) -55.6% 3.0ms (20.0ms) -85.0%
Q5 57.0ms (55.0ms) +3.6% 42.0ms (48.0ms) -12.5% 5.0ms (20.5ms) -75.6%
Q6 201.0ms (117.0ms) +71.8% 71.0ms (114.5ms) -38.0% 19.0ms (82.5ms) -77.0%
Q7 1.0ms (1.0ms) +0.0% 0.000ms (0.000ms) +0.0% 0.000ms (0.000ms) +0.0%
Q8 8.0ms (13.0ms) -38.5% 5.0ms (6.0ms) -16.7% 10.5ms (6.5ms) +61.5%
Q9 79.0ms (82.0ms) -3.7% 68.0ms (77.5ms) -12.3% 7.0ms (39.0ms) -82.1%
Q10 95.0ms (84.0ms) +13.1% 66.5ms (85.0ms) -21.8% 6.0ms (59.0ms) -89.8%
Q11 52.0ms (22.0ms) +136.4% 34.5ms (23.0ms) +50.0% 87.0ms (30.0ms) +190.0%
Q12 56.0ms (33.0ms) +69.7% 36.5ms (25.0ms) +46.0% 86.0ms (34.5ms) +149.3%
Q13 236.0ms (120.0ms) +96.7% 90.5ms (114.0ms) -20.6% 77.0ms (84.0ms) -8.3%
Q14 526.0ms (149.0ms) +253.0% 129.0ms (145.5ms) -11.3% 69.0ms (109.5ms) -37.0%
Q15 287.0ms (116.0ms) +147.4% 88.5ms (107.0ms) -17.3% 76.5ms (102.5ms) -25.4%
Q16 116.0ms (111.0ms) +4.5% 96.0ms (102.5ms) -6.3% 6.0ms (21.5ms) -72.1%
Q17 521.0ms (227.0ms) +129.5% 233.0ms (219.5ms) +6.2% 65.0ms (105.0ms) -38.1%
Q18 516.0ms (224.0ms) +130.4% 232.5ms (225.5ms) +3.1% 68.0ms (109.5ms) -37.9%
Q19 908.0ms (456.0ms) +99.1% 374.5ms (404.0ms) -7.3% 97.0ms (152.0ms) -36.2%
Q20 14.0ms (11.0ms) +27.3% 4.5ms (10.5ms) -57.1% 11.5ms (17.5ms) -34.3%
Q21 1.47s (216.0ms) +581.5% 798.5ms (207.5ms) +284.8% 303.0ms (333.5ms) -9.1%
Q22 1.79s (207.0ms) +764.7% 1.04s (198.5ms) +422.7% 183.5ms (410.5ms) -55.3%
Q23 4.72s (563.0ms) +737.8% 2.75s (557.0ms) +392.9% 631.0ms (877.5ms) -28.1%
Q24 29.04s (1.04s) +2687.3% 1.88s (1.04s) +80.2% 643.0ms (2.84s) -77.4%
Q25 296.0ms (85.0ms) +248.2% 18.0ms (61.5ms) -70.7% 42.0ms (122.5ms) -65.7%
Q26 141.0ms (49.0ms) +187.8% 29.5ms (49.5ms) -40.4% 75.0ms (86.5ms) -13.3%
Q27 293.0ms (66.0ms) +343.9% 26.0ms (65.0ms) -60.0% 67.5ms (125.0ms) -46.0%
Q28 1.44s (262.0ms) +447.7% 1.61s (256.5ms) +527.7% 262.0ms (345.0ms) -24.1%
Q29 2.10s (997.0ms) +110.2% 2.10s (1.00s) +109.4% 514.5ms (414.0ms) +24.3%
Q30 31.0ms (25.0ms) +24.0% 19.5ms (27.5ms) -29.1% 7.0ms (22.5ms) -68.9%
Q31 632.0ms (105.0ms) +501.9% 80.5ms (107.0ms) -24.8% 72.5ms (149.0ms) -51.3%
Q32 1.00s (109.0ms) +817.4% 167.0ms (106.0ms) +57.5% 107.5ms (154.5ms) -30.4%
Q33 355.0ms (344.0ms) +3.2% 309.5ms (331.0ms) -6.5% 9.0ms (71.0ms) -87.3%
Q34 1.53s (450.0ms) +239.1% 970.5ms (443.0ms) +119.1% 178.0ms (334.0ms) -46.7%
Q35 1.51s (436.0ms) +247.2% 956.0ms (440.5ms) +117.0% 165.5ms (325.5ms) -49.2%
Q36 99.0ms (100.0ms) -1.0% 83.5ms (92.0ms) -9.2% 8.0ms (21.0ms) -61.9%
Q37 362.0ms (123.0ms) +194.3% 95.5ms (113.5ms) -15.9% 31.5ms (86.0ms) -63.4%
Q38 87.0ms (54.0ms) +61.1% 30.0ms (49.0ms) -38.8% 18.5ms (27.0ms) -31.5%
Q39 329.0ms (58.0ms) +467.2% 38.0ms (57.0ms) -33.3% 17.0ms (86.0ms) -80.2%
Q40 854.0ms (216.0ms) +295.4% 323.0ms (215.0ms) +50.2% 71.5ms (153.0ms) -53.3%
Q41 29.0ms (21.0ms) +38.1% 11.5ms (20.0ms) -42.5% 8.0ms (18.5ms) -56.8%
Q42 22.0ms (19.0ms) +15.8% 10.5ms (20.5ms) -48.8% 8.5ms (18.5ms) -54.1%
Q43 18.0ms (15.0ms) +20.0% 13.0ms (14.5ms) -10.3% 11.0ms (10.0ms) +10.0%

⚠️ LiquidCache is slower on 12 queries (warm)

  • Q28: warm +527.7% (1.61s vs 256.5ms)
  • Q22: warm +422.7% (1.04s vs 198.5ms)
  • Q23: warm +392.9% (2.75s vs 557.0ms)
  • Q21: warm +284.8% (798.5ms vs 207.5ms)
  • Q34: warm +119.1% (970.5ms vs 443.0ms)
  • Q35: warm +117.0% (956.0ms vs 440.5ms)
  • Q29: warm +109.4% (2.10s vs 1.00s)
  • Q24: warm +80.2% (1.88s vs 1.04s)
  • Q32: warm +57.5% (167.0ms vs 106.0ms)
  • Q40: warm +50.2% (323.0ms vs 215.0ms)
  • Q11: warm +50.0% (34.5ms vs 23.0ms)
  • Q12: warm +46.0% (36.5ms vs 25.0ms)

Compared Liquid vs DataFusionDefault on the same runner
Regressions: warm-time increases of at least 15%. Cold Time: first iteration; Warm Time: median of remaining iterations.

A row group's snapshot map kept arrays alive after the cache evicted
them, so the memory counter dropped while the memory stayed.

- The reader drops each batch's snapshots once it has emitted or skipped
  the batch, instead of at the end of the row group.
- Core records the snapshots that prefetch hands out. A memory entry that
  leaves the index while one of its snapshots is alive stays charged until
  the snapshot is dropped. Usage reads and refused reservations release
  charges whose snapshots are gone.
- The first row group's prefetch waits for headroom like later row groups.
  A prefetch stops when the cache refuses an insert, rather than keeping
  an uncounted copy, and snapshots a disk entry only if hydration puts it
  back in memory.

Closes #79
@rohan-hotdata
rohan-hotdata merged commit f81a03d into main Nov 18, 2026
14 checks passed
anoop-narang added a commit that referenced this pull request Nov 18, 2026
Resolve against the snapshot accounting from #104: sweep_held releases
through the saturating release, and try_reserve_memory sweeps held
snapshots and retries once when the reservation is refused.

Claude-Session: https://claude.ai/code/session_01LbJrgYNLF7FxwMy5VSHXar
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reader: row-group snapshots pin batches outside the budget

1 participant