Skip to content

cache: hydrated Arrow entries are charged about 2x #75

Open

Description

@zfarrell

Found by a memory-accounting audit of 8b3c813. The tests are on branch audit/memory-tests and fail on main today; check out that branch to run them. File and line links point at that commit.

What goes wrong.

  • read_disk_arrow_array (core.rs:1123-1137) decodes through the IPC StreamReader, so every buffer of the result slices one message allocation.
  • The charge is get_array_memory_size() (src/core/src/cache/cached_batch.rs:63), which counts that allocation once per buffer.
  • This over-counts, so the cache evicts more than it needs to. It affects the types the liquid encoder rejects (Boolean, timestamps with a timezone, LargeUtf8, List/Struct/Map, some dictionaries) and any test that uses the Evict policy.

Evidence.

Column Before the spill After hydration Ratio
Nullable Int32, 8,192 rows 33,888 67,680 2.0x
Nullable string 164,988 309,624 1.88x
Utf8 high-card, 64 arrays (allocator run) 56.97 MB charged 28.52 MB live 2.0x

The 1.88x string ratio is not fully explained by the per-buffer mechanism, which predicts about 3x. The mechanism is inferred, not verified.

Tests.

Test File Command Flags
audit_disk_round_trip_keeps_the_charge_of_a_nullable_int32 src/core/src/cache/core.rs:4112 cargo +1.96.0 test -p liquid-cache --lib audit_disk_round_trip_keeps_the_charge_of_a_nullable_int32 Inline
audit_disk_round_trip_keeps_the_charge_of_a_nullable_string src/core/src/cache/core.rs:4124 cargo +1.96.0 test -p liquid-cache --lib audit_disk_round_trip_keeps_the_charge_of_a_nullable_string

Inline test (the helper plus one case; uses the existing assert_memory_budget_matches_index):

    async fn audit_disk_round_trip_charge(array: ArrayRef) -> (usize, usize) {
        let cache = LiquidCacheBuilder::new()
            .with_max_memory_bytes(1 << 24)
            .with_max_disk_bytes(1 << 24)
            .with_eviction_policy(Box::new(crate::cache::Evict))
            .with_cache_policy(Box::new(LiquidPolicy::new()))
            .with_hydration_policy(Box::new(AlwaysHydrate::new()))
            .build()
            .await;
        let entry = EntryID::from(1020usize);
        cache.insert(entry, 0, array.clone()).await.unwrap();
        let before = cache.budget().memory_usage_bytes();
        cache.flush_all_to_disk().await.unwrap();
        assert_eq!(cache.budget().memory_usage_bytes(), 0);
        let read = cache.get(&entry, 0).await.expect("present");
        assert_eq!(read.as_ref(), array.as_ref());
        assert_memory_budget_matches_index(&cache);
        (before, cache.budget().memory_usage_bytes())
    }

    #[tokio::test]
    async fn audit_disk_round_trip_keeps_the_charge_of_a_nullable_int32() {
        let array: ArrayRef = Arc::new(Int32Array::from_iter(
            (0..8192).map(|i| (i % 7 != 0).then_some(i)),
        ));
        let (before, after) = audit_disk_round_trip_charge(array).await;
        assert!(
            after <= before + before / 10,
            "hydrated entry charged {after} bytes, {before} before the spill"
        );
    }

(The branch's tier matches! assertions are dropped here for brevity.)

Flags: P probabilistic (a pass after a fix only means the race wasn't hit), F interleaving forced by a test hook, L Linux only, D needs Docker, M measurement only, C control (passes by design).

Activity

  1. added
    bugSomething isn't working
    performanceWasted work, over-eviction or excess IO
    on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingperformanceWasted work, over-eviction or excess IO

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions