Skip to content

write_batch_to_disk reserves the full length when replacing an existing object #46

Open

Description

@zfarrell

Follow-up from #45 review. write_batch_to_disk reserves len before the put while the previous copy still holds previous.bytes, and releases the old reservation only after the reserve succeeds. Replacing an object on a nearly full disk tier therefore needs both sizes free at once; the reserve fails, CacheFull propagates, and flush_all_to_disk drops an entry whose rewrite would not have grown the tier (flush_dropping_hydrated_entry_releases_its_disk_copy shows that outcome).

Fix: read the record before the reserve loop, reserve len - previous.bytes when the value grows and release previous.bytes - len after the put when it shrinks, keeping the record in place if the reserve fails so discard_disk_copy still matches a live reservation. The flush test then needs a different route to CacheFull.

Activity

  1. zfarrell commented on Nov 17, 2026

    @zfarrell
    Author

    Found by a memory-accounting audit of 8b3c813. The tests are on branch audit/memory-tests and fail on main today; check out that branch to run them. File and line links point at that commit.

    This adds the test that #46 asks for.

    • With a disk tier that fits exactly two records, re-inserting one key at the same size reserved a second 1,488 bytes while the original was still charged.
    • That evicted the other record (1 failed disk reservation). The insert returned Ok(()).
    • With LiquidPolicy, the outcome is a needless eviction rather than CacheFull.
    • The related claim that "hydrate then re-flush needs two slots" does not reproduce, because hydration already releases the disk charge.
    Test File Command Flags
    audit_rewriting_a_disk_record_in_place_evicts_no_other_record src/core/src/cache/core.rs:3812 cargo +1.96.0 test -p liquid-cache --lib audit_rewriting_a_disk_record_in_place_evicts_no_other_record -- --nocapture needs the branch's audit_cache helper
    audit_respilling_a_hydrated_entry_fits_a_disk_tier_of_one_object src/core/src/cache/core.rs:3775 cargo +1.96.0 test -p liquid-cache --lib audit_respilling_a_hydrated_entry_fits_a_disk_tier_of_one_object -- --nocapture C (passes; disproves the variant)

    Flags: P probabilistic (a pass after a fix only means the race wasn't hit), F interleaving forced by a test hook, L Linux only, D needs Docker, M measurement only, C control (passes by design).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions