Skip to content

cache: reset() leaves objects, policy entries and in-flight charges #73

Open

Description

@zfarrell

Found by a memory-accounting audit of 8b3c813. The tests are on branch audit/memory-tests and fail on main today; check out that branch to run them. File and line links point at that commit.

What goes wrong.

  • LiquidCache::reset (core.rs:376-379) only clears the index and zeroes both counters. It does not:
    • delete store objects;
    • clear the policy queues;
    • fence work that is in flight.
  • It is reachable on a live server through the admin /reset_cache handler (src/datafusion-server/src/admin_server/handlers.rs:50-55). The unsafe doc on src/datafusion/src/cache/mod.rs:532 says to call it only when nothing else uses the cache, but nothing enforces that.
  • A spill that finishes or is dropped after the reset wraps the disk counter (cache: budget counters wrap below zero #72), and that disables the disk tier.

Evidence.

Case Result
Spill then reset() The store still holds 1 object that no record names
Server /reset_cache Reports 0 disk usage while 16 objects remain (593,920-byte file)
Spill dropped after reset() Disk usage 18,446,744,073,709,550,128; a 528-byte spill is then refused with CacheFull
Spill that installs its record after reset() Usage 0 while the index charges 1,488; removing the record wraps the counter
Queues after reset() The policy still offers 4 memory and 6 disk victims for an empty index
reset() racing inserts Budget 5 entries short of the index

Tests.

Test File Command Flags
audit_reset_deletes_disk_objects src/core/src/cache/core.rs:4063 cargo +1.96.0 test -p liquid-cache --lib audit_reset_deletes_disk_objects Inline
audit_reset_under_a_dropped_spill_keeps_the_disk_counter_in_bounds src/core/src/cache/core.rs:3185 cargo +1.96.0 test -p liquid-cache --lib audit_reset_under_a_dropped_spill -- --nocapture
audit_reset_under_a_finishing_spill_leaves_its_record_charged src/core/src/cache/core.rs:3241 cargo +1.96.0 test -p liquid-cache --lib audit_reset_under_a_finishing_spill -- --nocapture
audit_reset_leaves_the_policy_no_victims src/core/src/cache/core.rs:3278 cargo +1.96.0 test -p liquid-cache --lib audit_reset_leaves_the_policy_no_victims -- --nocapture
audit_reset_racing_inserts_leaves_the_memory_budget_matching_the_index src/core/src/cache/core.rs:3319 cargo +1.96.0 test -p liquid-cache --lib audit_reset_racing_inserts -- --nocapture P
audit_reset_cache_handler_empties_the_disk_it_reports_as_empty src/datafusion-server/src/admin_server/handlers_audit_tests.rs:48 cargo +1.96.0 test -p liquid-cache-datafusion-server --lib audit_reset_cache_handler_empties_the_disk_it_reports_as_empty -- --nocapture

Inline test (uses the existing flushed and assert_disk_matches_index):

    #[tokio::test]
    async fn audit_reset_deletes_disk_objects() {
        let entry = EntryID::from(1010usize);
        let array: ArrayRef = Arc::new(Int32Array::from_iter_values(0..256));
        let (cache, store, _directory) = flushed(entry, &array).await;
        assert_eq!(store.len().await.unwrap(), 1);

        cache.reset();

        assert!(!cache.contains(&entry, 0));
        assert_eq!(cache.budget.disk_usage_bytes(), 0);
        assert_disk_matches_index(&cache, &store).await;
    }

Flags: P probabilistic (a pass after a fix only means the race wasn't hit), F interleaving forced by a test hook, L Linux only, D needs Docker, M measurement only, C control (passes by design).

Activity

  1. added
    bugSomething isn't working
    memoryMemory accounting or memory usage
    on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingmemoryMemory accounting or memory usage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions