Repository navigation
Description
Native macOS is not supported today. This ticket records what I measured, and what I think it means. It is not a plan.
Test system: aarch64-apple-darwin, rustc 1.94.1 stable.
Evidence: the build
cargo check --workspace --all-targets fails in one crate only.
The benchmark crate uses perf-event2 (benchmark/Cargo.toml:44). This library needs perf_event_open. Only Linux has this system call. The build gives 6 errors.
Two other libraries in the same crate do build on macOS. I tested usdt 0.6 and pprof 0.15 in a separate crate. Both compile.
The dev-tools crate also fails, but it fails for the generated assets/tailwind.css file. This happens on Linux too. It is not related to the platform.
Evidence: the tests
| Crate | Pass | Fail |
|---|---|---|
| liquid-cache | 218 | 18 |
| liquid-cache-common | 8 | 0 |
| liquid-cache-datafusion | 41 | 21 |
| liquid-cache-datafusion-client | 28 | 0 |
| liquid-cache-datafusion-local | 31 | 1 |
| liquid-cache-datafusion-server | 3 | 12 |
| doc-tests | — | 6 |
57 of the 58 failures give the same message:
InvalidArgument("direct_io not supported on target_os")
t4::MountOptions::default() sets direct_io: true (t4-0.1.7 src/store.rs:28-34). t4 gives an error for this option on all systems that are not Linux (src/store.rs:60-67).
Two of these failures are not test-only:
src/core/src/cache/builders.rs:140—LiquidCacheBuilder::build()stops the program. This is the README example.examples/core.rs:10also uses this path.src/datafusion-server/src/service.rs:61— the Flight server stops at start.
There are 15 t4::mount( calls in 10 files.
src/datafusion-local/src/lib.rs:192-197 does not use the default. It sets direct_io: cfg!(target_os = "linux"). Its 31 tests pass. This shows the buffered I/O path of t4 operates correctly on macOS. It writes to disk, reads from disk, and removes entries.
Evidence: one different failure
liquid-cache-datafusion-local tests::test_provide_schema2 fails for a different reason. It compares usage.memory_bytes with a recorded value. macOS gives a value 935 bytes lower at all three points (1000915 → 999980, 1036304 → 1035369).
The value is the same in each run. The temporary directory name is not the cause. I ran the test again with TMPDIR=/tmp and got the same value.
Six other snapshots record an exact usage.memory_bytes. All six pass on macOS. Four of them write to disk and read from disk:
| Snapshot | memory_bytes | disk_bytes |
|---|---|---|
| referer_filtering | 917947 | 877216 |
| url_selection_and_ordering | 189553 | 706252 |
| os_selection | 954618 | 164536 |
| url_prefix_filtering | 912473 | 139416 |
Hypotheses
The 935-byte difference relates to the data, not to the platform. test_provide_schema2 is the only test that reads dev/test_parquet/openobserve.parquet (tests/mod.rs:32). All other tests read nano_hits.parquet.
I removed three other possible causes:
- Not the data type sizes. Both processors use the same sizes.
- Not the memory allocator. Arrow makes buffer sizes a multiple of 64. 935 is not a multiple of 64. A counter that uses
len()is more probable, for examplesrc/core/src/liquid_array/raw/fsst_buffer.rs:134-136. - Not the I/O mode. The four snapshots above use buffered I/O on macOS and direct I/O on Linux. They give the same values.
A change to buffered I/O has a cost that is not visible. The operating system then keeps a second copy of the data in its own cache. The admission gate assumes that counted bytes are bytes in memory (src/datafusion/src/optimizers/mod.rs:58). This is not correct with buffered I/O. A developer can measure incorrect behavior on a Mac.
A quiet fallback is a risk. README.md:96 tells the user that direct I/O makes LiquidCache look slower in development. On macOS the opposite is true.
Three code paths are different on macOS. Two are not tested today.
- t4 has two I/O methods (
t4/src/io/io_worker.rs:11-15). Linux usesio_uring.rs. All other systems usegeneric.rs.generic.rshas unsafe code (unsafe impl Send,from_raw_parts_mut). No job in CI runs it. src/datafusion/src/utils.rs:75-79uses a BMI2 function on x86_64. This is unsafe code on the filter path. The two tests that compare it with the standard function are#[cfg(target_arch = "x86_64")](utils.rs:316,:359). They do not run on ARM.
t4::mount() is the origin of the problem. Its default option set cannot mount on the platform that compiled it. A correction in t4 (fcntl(F_NOCACHE) is the macOS equivalent of O_DIRECT) would remove the problem for all 15 call sites and keep the single-copy property. t4 is a different repository.
Not a problem
t4::PAGE_SIZE is a constant 4096 (t4/src/lib.rs:28). It does not read the system page size. The 16 KB pages of Apple Silicon do not make store files incompatible.
Open questions
- Do we support macOS for development only, or also for the published crates?
- If a fallback to buffered I/O is acceptable, is it quiet or does it give a warning?
- Is a Linux virtual machine a better answer for developers who need correct performance data?
Activity