Skip to content

cache: restart leaves prior objects uncharged and unreachable #95

Open

Description

@zfarrell

Found by a memory-accounting audit of 8b3c813. The tests are on branch audit/memory-tests and fail on main today; check out that branch to run them. File and line links point at that commit.

What goes wrong. The builders mount a fixed liquid_cache.t4 path with no truncation or reconciliation (src/datafusion-local/src/lib.rs:196, src/datafusion-server/src/service.rs:61). On a persistent directory, t4 replays every earlier object as live, but the new cache's index is empty and its budget starts at 0.

Evidence.

  • 1,000 objects (4.1 MB of file) survived a remount with 0 bytes charged, and the new cache served 0 of 1,000 gets.
  • In the second test: 8 objects, 0 indexed, 0 charged.

Tests.

Test File Command Flags
claim4_restart_orphans src/core/tests/disk_tier_audit.rs:448 cargo +1.96.0 test -p liquid-cache --test disk_tier_audit claim4 -- --nocapture --test-threads=1
audit_cache_rebuilt_over_an_existing_file_accounts_for_its_objects src/core/src/cache/builders_audit_tests.rs:205 cargo +1.96.0 test -p liquid-cache --lib audit_cache_rebuilt_over_an_existing_file_accounts_for_its_objects -- --nocapture

Flags: P probabilistic (a pass after a fix only means the race wasn't hit), F interleaving forced by a test hook, L Linux only, D needs Docker, M measurement only, C control (passes by design).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions