Repository navigation
Open
Labels
bugSomething isn't workingSomething isn't workingperformanceWasted work, over-eviction or excess IOWasted work, over-eviction or excess IO
Description
Found by a memory-accounting audit of 8b3c813. The tests are on branch audit/memory-tests and fail on main today; check out that branch to run them. File and line links point at that commit.
What goes wrong.
- A disk read hydrates by calling
insert_inner(core.rs:917). - When the entry cannot fit in memory and the policy offers no memory victim,
insert_innerwrites the entry back to disk (core.rs:559-568). That rewrites the object the read just loaded. - Every such read costs a full disk write, a serialized copy, and WAL growth (cache: t4 file outgrows the disk budget #93).
Evidence.
- With a 64-byte memory budget and a 1,024-row Int32 entry, two reads caused two writes; zero were expected.
- Under production defaults with an incompressible array: one write per read, 5 of 5.
- The original
0..1024data does not reproduce underTranscodeEvict, because it compresses to about 40 bytes and stays in memory.
Tests.
| Test | File | Command | Flags |
|---|---|---|---|
audit_a_read_that_cannot_hydrate_writes_nothing |
src/core/src/cache/core.rs:4138 |
cargo +1.96.0 test -p liquid-cache --lib audit_a_read_that_cannot_hydrate_writes_nothing |
Inline |
audit_prod_unhydratable_read_writes_nothing_sequential |
src/core/src/cache/core.rs:3106 |
cargo +1.96.0 test -p liquid-cache --lib audit_prod_unhydratable_read_writes_nothing_sequential -- --nocapture |
|
audit_prod_unhydratable_read_writes_nothing_concurrent |
src/core/src/cache/core.rs:3115 |
cargo +1.96.0 test -p liquid-cache --lib audit_prod_unhydratable_read_writes_nothing_concurrent -- --nocapture |
Inline test:
#[tokio::test]
async fn audit_a_read_that_cannot_hydrate_writes_nothing() {
let directory = tempfile::tempdir().unwrap();
let store = t4::mount(directory.path().join("cache.t4")).await.unwrap();
let cache = LiquidCacheBuilder::new()
.with_max_memory_bytes(64)
.with_max_disk_bytes(1 << 20)
.with_eviction_policy(Box::new(crate::cache::Evict))
.with_cache_policy(Box::new(LiquidPolicy::new()))
.with_hydration_policy(Box::new(AlwaysHydrate::new()))
.with_store(store.clone())
.build()
.await;
let entry = EntryID::from(1030usize);
let array: ArrayRef = Arc::new(Int32Array::from_iter_values(0..1024));
cache.insert(entry, 0, array.clone()).await.unwrap();
assert!(matches!(
cache.index().get(&entry).unwrap().entry,
CacheEntry::DiskArrow { .. }
));
let inserting = cache.observer().runtime_snapshot().write_io_count;
assert_eq!(inserting, 1, "the insert spilled the entry once");
for _ in 0..2 {
let read = cache.get(&entry, 0).await.expect("present");
assert_eq!(read.as_ref(), array.as_ref());
}
assert!(matches!(
cache.index().get(&entry).unwrap().entry,
CacheEntry::DiskArrow { .. }
));
assert_disk_matches_index(&cache, &store).await;
assert_eq!(
cache.observer().runtime_snapshot().write_io_count,
0,
"reads rewrote the object they had just read"
);
}Flags: P probabilistic (a pass after a fix only means the race wasn't hit), F interleaving forced by a test hook, L Linux only, D needs Docker, M measurement only, C control (passes by design).
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingperformanceWasted work, over-eviction or excess IOWasted work, over-eviction or excess IO
Activity