Skip to content

Take few rows of encoded Dict values when appending to a builder - #11545

Merged
joseph-isaacs merged 1 commit into
ji/alp-rd-take-chunked-benchfrom
ji/dict-append-lazy-take
Nov 17, 2026
Merged

joseph-isaacs merged 1 commit into
ji/alp-rd-take-chunked-benchfrom
ji/dict-append-lazy-take

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR stacks on #11548, which adds the alp_rd_take_chunked benchmark on its own so that this change can be measured against develop. Merge #11548 first, then retarget this PR to develop.

array.take(indices) on an encoded array is a lazy Dict whose values are the whole encoded array. Executing that Dict directly is fine, because the values' take kernel runs as an execute-parent kernel. But when the Dict sits inside a ChunkedArray, canonicalizing appends each chunk to a builder, and Dict::append_to_builder decoded every value before gathering.

Here is where that showed up: a vector re-rank query took 1–3 rows from each 4096-row FixedSizeList<f32, 64> ALP-RD segment, chunked across segments. Each 262,144-float segment was decoded in full. In pprof, 77% of the query was in vortex_alp::alp_rd::fused::decode under Dict::append_to_builder, and the query took 8.6 ms against 1.8 ms when reading slices instead.

Changes

Dict::append_to_builder already sent sparse Utf8/Binary selections over encoded values through the values' take kernel. This PR extends that branch to every dtype, so ALP-RD, BitPacked and the other take kernels decode only the selected rows. Dense dictionaries and decoded values keep the existing gather paths, including the VarBin fast paths.

The selection threshold now depends on the dtype (sparse_take_ratio):

  • Fixed-width types take when there are at most 1/16 as many codes as values. On ALP-RD f32, the take path costs about 13 ns per row, against about 0.8 ns per value to decode everything, so taking only wins below about one row in eleven. With a 1/4 threshold, taking 1/4 of the rows was 2.3× slower than decoding everything (2.95 ms vs 1.26 ms). The 1/16 threshold was measured only on ALP-RD with bit-packed children. Other encodings may break even at a different point, but this one is conservative for the case measured.
  • Utf8/Binary keep their existing ratio of 4. I did not measure them here.

Benchmark

Median times from alp_rd_take_chunked, which runs four ALP-RD f32 segments of 262,144 values:

rows taken per segment before after
FSL: 1 vector 830 µs 27 µs
FSL: 16 835 µs 94 µs
FSL: 256 (1/16 of the values) 947 µs 600–740 µs
FSL: 1024 / 2048 unchanged path unchanged path
primitive: 1 ~800 µs 13 µs
primitive: 2048 ~800 µs 125 µs

The machine these ran on varies by about ±30% between runs.

No other CPU benchmark reaches the changed branch. I checked by running every CPU benchmark crate once in test mode with a temporary marker on the branch, and only alp_rd_take_chunked hit it. The CUDA, DuckDB and Python benchmarks were not checked.

Tests

  • Added append_to_builder_takes_few_rows_of_encoded_values, which checks that a sparse take over chunked values, with a null code, appends the right values after a row already in the builder.
  • cargo test -p vortex-array -p vortex-alp and cargo clippy -p vortex-array -p vortex-alp --all-targets --all-features -- -D warnings pass. After rebasing onto the current develop, I re-ran the dict tests and clippy.
  • Formatted with cargo +nightly-2026-11-13 fmt.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UeniBE6jGwH9sRoqTmLZFz

@codspeed

codspeed Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚡ 4 improved benchmarks
❌ 1 regressed benchmark
✅ 1964 untouched benchmarks
⏩ 53 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Walltime decode_bitpacked_chunked_avx512[u32, 524288] 17.4 µs 19.5 µs -10.58%
⚡ CPU Simulation primitive[1] 864.1 µs 163.1 µs ×5.3
⚡ CPU Simulation primitive[16] 865.3 µs 193.8 µs ×4.5
⚡ CPU Simulation fixed_size_list[1] 979.6 µs 366 µs ×2.7
⚡ CPU Simulation fixed_size_list[16] 983.6 µs 538.3 µs +82.72%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ji/dict-append-lazy-take (0851174) with ji/alp-rd-take-chunked-bench (6a5c3c4)

Open in CodSpeed

Footnotes

  1. 53 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@robert3005

Copy link
Copy Markdown
Contributor

can you split the benchmarks so we can see the improvement?

`array.take(indices)` on an encoded array is a lazy `Dict` whose values are
the whole encoded array. When such a Dict sits in a ChunkedArray,
canonicalizing appends each chunk to a builder, and `Dict::append_to_builder`
decoded every value before gathering. Taking a few vectors from each
262,144-float ALP-RD segment decoded every segment in full.

`append_to_builder` already executed sparse Utf8/Binary selections through
the take kernel of the values. Do the same for every dtype, so ALP-RD,
BitPacked and the other take kernels decode only the selected rows. For
fixed-width dtypes, take when the codes are at most 1/16 of the values:
the take kernels cost over ten times as much per row as a bulk decode per
value, and taking rows of an ALP-RD f32 segment breaks even at about one
row in eleven. Utf8/Binary keep their ratio of 4.

In the `alp_rd_take_chunked` benchmark, one vector per segment drops from
about 830 us to 27 us, and 1 to 2,048 primitive rows per segment from
about 800 us to 13-130 us.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBE6jGwH9sRoqTmLZFz
@joseph-isaacs
joseph-isaacs force-pushed the ji/dict-append-lazy-take branch from ddae909 to 0851174 Compare November 17, 2026 15:44
@joseph-isaacs
joseph-isaacs changed the base branch from develop to ji/alp-rd-take-chunked-bench November 17, 2026 15:44
@joseph-isaacs
joseph-isaacs merged commit 91bca2f into ji/alp-rd-take-chunked-bench Nov 17, 2026
107 of 113 checks passed
@joseph-isaacs
joseph-isaacs deleted the ji/dict-append-lazy-take branch November 17, 2026 16:58
joseph-isaacs added a commit that referenced this pull request Nov 17, 2026
Fold it into the Pull Requests section that develop added, and add what
landing #11548 and #11545 showed: the benchmark PR must also count its
benchmarks in `scripts/codspeed_bench_counts.json`, and merging the stacked
change while its base is still the benchmark branch folds it into the
benchmark PR.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBE6jGwH9sRoqTmLZFz
joseph-isaacs added a commit that referenced this pull request Nov 17, 2026
…array (#11548)

## Summary

This PR adds only a benchmark, with no change in behavior. It is split
out of #11545 so that #11545 can be measured against `develop`. #11545
stacks on this PR.

`array.take(indices)` on an encoded array is a lazy `Dict` whose values
are the whole encoded array. When that `Dict` is a chunk of a
`ChunkedArray`, canonicalizing appends it to a builder through
`Dict::append_to_builder`. On `develop`, that path decodes every value
of a non-string `Dict` before gathering. In a vector re-rank query, it
decoded each 262,144-float ALP-RD segment in full to read 1–3 vectors.


Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants