Repository navigation
Dispatch structural kernels by physical width - #10841
Conversation
Merging this PR will regress 3 benchmarks4 warnings, 1 note
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.973x ➖, 1↑ 0↓)
datafusion / parquet / ns (0.995x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.016x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.997x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.001x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.998x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.996x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.001x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.994x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.999x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.004x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.005x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.001x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.998x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.998x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.002x ➖, 1↑ 1↓)
No file size changes detected. |
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.996x ➖, 1↑ 0↓)
datafusion / parquet / ns (0.995x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.006x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.002x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.984x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.963x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.958x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.005x ➖, 0↑ 0↓)
|
|
Whoops the regressions are real, making a fix |
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
The width dispatch change made the primitive interleave gather yield `Result<[u8; W], VortexError>` per row. `[u8; W]` has alignment 1, so the `Ok` payload sits at offset 1 and LLVM splits each 8-byte copy into 4 + 2 + 1 + 1 byte moves. This roughly doubled the time of the `interleave` primitive benchmarks. Write rows through the buffer's spare capacity slice instead. This keeps each copy a single move and lets LLVM hold the loop state in registers. The benchmarks now match or beat the code before the width dispatch change. Claude-Session: https://claude.ai/code/session_019amF7o3i2tJvRGbgL3c3GA Signed-off-by: Claude <noreply@anthropic.com>
2ea4cfd to
f0517af
Compare
|
Oh claude already made the fix, the microbenchmarks aren't up to date |
|
Somehow some things are faster, not sure I believe that |
Summary
Follow-up to #10740. Kernels that only copy or compare stored bits can share code across signed, unsigned, and floating-point types of the same width.
Changes
API Changes
match_each_ptype_width!,match_each_decimal_type_width!, andmatch_each_power_of_two_width!bind aconst W: usize, with an explicit fallback for unsupported widths and an optional specialization list.Validation: pinned nightly formatting, scoped Clippy with all targets and features and
-D warnings, 100 focused tests, and seven width doctests pass.Code size comparison
Compared base
a59294e35ewith77a96de1c0using separate checkouts and target directories. Both use rustc 1.99.1, LLVM 23.1.1,aarch64-apple-darwin, the release profile (opt-level 3, one codegen unit, LTO off), package default features, and the repository's frame-pointer flag.Machine-code bytes from Mach-O
__textsections:vortex-bufferlibraryvortex-arraylibraryvortex-fastlaneslibraryvxbinaryThe linked
vxfile also shrinks from 120,844,528 to 120,779,936 bytes (-64,592). Library totals exclude Rust metadata. Small archive increases invortex-bufferandvortex-fastlanescome from metadata, while their machine sections are unchanged.Commands run in each checkout:
These measurements cover this target and profile. Runtime benchmarks were not run locally.