Repository navigation
Take few rows of encoded Dict values when appending to a builder - #11545
Conversation
Merging this PR will regress 1 benchmark1 warning
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
|
can you split the benchmarks so we can see the improvement? |
`array.take(indices)` on an encoded array is a lazy `Dict` whose values are the whole encoded array. When such a Dict sits in a ChunkedArray, canonicalizing appends each chunk to a builder, and `Dict::append_to_builder` decoded every value before gathering. Taking a few vectors from each 262,144-float ALP-RD segment decoded every segment in full. `append_to_builder` already executed sparse Utf8/Binary selections through the take kernel of the values. Do the same for every dtype, so ALP-RD, BitPacked and the other take kernels decode only the selected rows. For fixed-width dtypes, take when the codes are at most 1/16 of the values: the take kernels cost over ten times as much per row as a bulk decode per value, and taking rows of an ALP-RD f32 segment breaks even at about one row in eleven. Utf8/Binary keep their ratio of 4. In the `alp_rd_take_chunked` benchmark, one vector per segment drops from about 830 us to 27 us, and 1 to 2,048 primitive rows per segment from about 800 us to 13-130 us. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBE6jGwH9sRoqTmLZFz
ddae909 to
0851174
Compare
91bca2f
into
ji/alp-rd-take-chunked-bench
Fold it into the Pull Requests section that develop added, and add what landing #11548 and #11545 showed: the benchmark PR must also count its benchmarks in `scripts/codspeed_bench_counts.json`, and merging the stacked change while its base is still the benchmark branch folds it into the benchmark PR. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBE6jGwH9sRoqTmLZFz
…array (#11548) ## Summary This PR adds only a benchmark, with no change in behavior. It is split out of #11545 so that #11545 can be measured against `develop`. #11545 stacks on this PR. `array.take(indices)` on an encoded array is a lazy `Dict` whose values are the whole encoded array. When that `Dict` is a chunk of a `ChunkedArray`, canonicalizing appends it to a builder through `Dict::append_to_builder`. On `develop`, that path decodes every value of a non-string `Dict` before gathering. In a vector re-rank query, it decoded each 262,144-float ALP-RD segment in full to read 1–3 vectors. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Summary
This PR stacks on #11548, which adds the
alp_rd_take_chunkedbenchmark on its own so that this change can be measured againstdevelop. Merge #11548 first, then retarget this PR todevelop.array.take(indices)on an encoded array is a lazyDictwhose values are the whole encoded array. Executing thatDictdirectly is fine, because the values' take kernel runs as an execute-parent kernel. But when theDictsits inside aChunkedArray, canonicalizing appends each chunk to a builder, andDict::append_to_builderdecoded every value before gathering.Here is where that showed up: a vector re-rank query took 1–3 rows from each 4096-row
FixedSizeList<f32, 64>ALP-RD segment, chunked across segments. Each 262,144-float segment was decoded in full. In pprof, 77% of the query was invortex_alp::alp_rd::fused::decodeunderDict::append_to_builder, and the query took 8.6 ms against 1.8 ms when reading slices instead.Changes
Dict::append_to_builderalready sent sparse Utf8/Binary selections over encoded values through the values' take kernel. This PR extends that branch to every dtype, so ALP-RD, BitPacked and the other take kernels decode only the selected rows. Dense dictionaries and decoded values keep the existing gather paths, including the VarBin fast paths.The selection threshold now depends on the dtype (
sparse_take_ratio):f32, the take path costs about 13 ns per row, against about 0.8 ns per value to decode everything, so taking only wins below about one row in eleven. With a 1/4 threshold, taking 1/4 of the rows was 2.3× slower than decoding everything (2.95 ms vs 1.26 ms). The 1/16 threshold was measured only on ALP-RD with bit-packed children. Other encodings may break even at a different point, but this one is conservative for the case measured.Benchmark
Median times from
alp_rd_take_chunked, which runs four ALP-RDf32segments of 262,144 values:The machine these ran on varies by about ±30% between runs.
No other CPU benchmark reaches the changed branch. I checked by running every CPU benchmark crate once in test mode with a temporary marker on the branch, and only
alp_rd_take_chunkedhit it. The CUDA, DuckDB and Python benchmarks were not checked.Tests
append_to_builder_takes_few_rows_of_encoded_values, which checks that a sparse take over chunked values, with a null code, appends the right values after a row already in the builder.cargo test -p vortex-array -p vortex-alpandcargo clippy -p vortex-array -p vortex-alp --all-targets --all-features -- -D warningspass. After rebasing onto the currentdevelop, I re-ran the dict tests and clippy.cargo +nightly-2026-11-13 fmt.🤖 Generated with Claude Code
https://claude.ai/code/session_01UeniBE6jGwH9sRoqTmLZFz