Skip to content

server: no memory or disk limit by default #89

Open

Description

@zfarrell

Found by a memory-accounting audit of 8b3c813. The tests are on branch audit/memory-tests and fail on main today; check out that branch to run them. File and line links point at that commit.

What goes wrong.

  • LiquidCacheService::new passes max_memory_bytes.unwrap_or(usize::MAX) for memory and a hard-coded usize::MAX for disk (src/datafusion-server/src/service.rs:67-68). With no flag set, the server has no limits, and disk stays unlimited even when memory is configured.
  • /cache_info (src/datafusion-server/src/admin_server/handlers.rs:129-142) reports that usize::MAX as the memory limit and has no max_disk_bytes field.

Evidence. Both memory and disk came out as 18,446,744,073,709,551,615 with no flag set. Disk was the same with memory set to 1 GiB, and /cache_info reported the same value.

Tests.

Test File Command Flags
audit_service_default_budgets_are_finite src/datafusion-server/src/tests/audit_config.rs:20 cargo +1.96.0 test -p liquid-cache-datafusion-server --lib audit_service_default_budgets_are_finite -- --nocapture Inline
audit_service_disk_budget_is_finite_when_memory_is_configured src/datafusion-server/src/tests/audit_config.rs:54 cargo +1.96.0 test -p liquid-cache-datafusion-server --lib audit_service_disk_budget_is_finite_when_memory_is_configured -- --nocapture
audit_cache_info_reports_a_real_memory_limit src/datafusion-server/src/admin_server/handlers_audit_tests.rs:90 cargo +1.96.0 test -p liquid-cache-datafusion-server --lib audit_cache_info_reports_a_real_memory_limit -- --nocapture
audit_system_info_reports_the_memory_the_server_may_use src/datafusion-server/src/admin_server/handlers_audit_tests.rs:113 cargo +1.96.0 test -p liquid-cache-datafusion-server --lib audit_system_info_reports_the_memory_the_server_may_use -- --nocapture M on macOS; the assertion is Linux-gated and was not run

Inline test:

#[tokio::test]
async fn audit_service_default_budgets_are_finite() {
    let dir = tempfile::tempdir().unwrap();
    let service = LiquidCacheService::new(
        LiquidCacheService::context().unwrap(),
        None,
        Some(dir.path().to_path_buf()),
        Box::new(LiquidPolicy::new()),
        Box::new(TranscodeEvict),
        Box::new(AlwaysHydrate::new()),
    )
    .await
    .unwrap();
    let memory = service.cache().max_memory_bytes();
    let disk = service.cache().max_disk_bytes();
    let mut unbounded = Vec::new();
    if memory == usize::MAX {
        unbounded.push("max_memory_bytes");
    }
    if disk == usize::MAX {
        unbounded.push("max_disk_bytes");
    }
    assert!(
        unbounded.is_empty(),
        "LiquidCacheService::new(ctx, None, ..) leaves {unbounded:?} at usize::MAX \
         (memory = {memory}, disk = {disk})"
    );
}

Flags: P probabilistic (a pass after a fix only means the race wasn't hit), F interleaving forced by a test hook, L Linux only, D needs Docker, M measurement only, C control (passes by design).

Activity

  1. added
    bugSomething isn't working
    memoryMemory accounting or memory usage
    on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingmemoryMemory accounting or memory usage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions