Repository navigation
Read single Delta values from their lane instead of the whole chunk - #11469
Conversation
Delta transposes each 1024-value chunk into lanes and encodes every lane against its own base, so a value depends only on its lane's base and the deltas of its lane up to its row: 8 to 64 deltas by integer width. A Delta point read decoded the whole chunk instead. A read now sums just that lane prefix when the deltas are plain or bit-packed on the host, with or without patches, reading the base through the bases array. A repeated probe still decodes and keeps a chunk once it reads from it a second time, so scans within a chunk cost what they did. Other delta encodings fall back to the chunk decode. This matters wherever something searches or slices sorted Delta-encoded indices, such as Sparse patch indices: each step of the binary search was a whole-chunk decode. One-off reads are about 14x faster, binary searches over a 256-chunk array about 4x, and StatPopGen with editions disabled compresses 26% faster (17.2 s to 12.7 s), with identical output. Adds a search_sorted benchmark over a sorted Delta array. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Merging this PR will regress 1 benchmark1 warning
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Move the lane arithmetic of Delta point reads into fastlanes (spiraldb/fastlanes#226): `untranspose` finds a value's position in the transposed chunk, `delta_lane_row` its lane, and `undelta_single` or `unchecked_undelta_pack_single` sum that lane's prefix. The fastlanes version unpacks the lane in order, which is faster than locating each delta's bits on its own. Patched deltas take the packed sum and correct it for each of the chunk's patches that falls in the lane up to the value's row. fastlanes is patched to the PR's commit until it is released. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
spiraldb/fastlanes#226 replaced its unrolled per-width single-element undelta with one safe runtime-width loop, which avoids about 176 KB of generated code. Patch fastlanes to that commit and call the safe `undelta_pack_single`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
The fastlanes PR branch was rebased, so its earlier commit may no longer be fetchable. The new head has the same code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
spiraldb/fastlanes#226 now adds a lane's packed deltas 64 bits at a time, which brings a single read close to the unrolled per-width speed in a few KB of code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
The fastlanes PR now keeps the small single-element undelta by default and adds an opt-in `unroll` feature for per-width unrolled copies. The API is unchanged, so Vortex builds the default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
It moves the rolled and unrolled single-element undelta into their own files, without changing the API. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
It shares the lane and row arithmetic between bitpacking and delta, without changing the API or generated code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
fastlanes 0.8.1 releases spiraldb/fastlanes#226, so require it and drop the git `[patch.crates-io]` entry. 0.8.1 also requires `RLE::decode_unchecked`'s index type to be `'static`. `decode_chunk_checked` passes its index type through, so it takes the same bound; the index types are primitive integers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
|
Can we trim down the benchmark |
A Delta point read over patched bit-packed deltas collected every patch of the chunk into a fresh `Vec` and then checked each one's lane, so a read cost O(patches in the chunk) plus an allocation, though the value depends on at most `T` deltas. The patch correction now reads the patch indices and values in place. It binary-searches the chunk's range of patches once, then either checks those patches or, when there are more of them than positions in the value's lane prefix, binary-searches each prefix position. Per one-off read under callgrind, 256K u32 values: with about 20% of deltas patched, 7,016 to 3,517 instructions; with no patches, 916 to 886. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
`probe_once_patched` and `probe_repeated_patched` mirror `probe_once` and `probe_repeated` over the same values bit-packed one bit narrower than they need, so about half of the deltas are patches. The existing probe benchmarks have none. Against the previous probe code, which copied a chunk's patches on every read, the patched one-off reads take 1.28 us rather than 5.76 us, and the repeated ones 6.55 us rather than 7.73 us. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Each lane read found the deltas and their patches in the array again:
slot lookups, unwrapping a slice of the deltas, and rebuilding the
patches. A repeated probe now resolves them on its first lane read and
keeps clones of the arrays in its state, so later reads skip that work.
One-off reads keep looking them up in the array, without cloning
anything: holding clones costs atomic reference counting, which a single
read does not earn back. Both paths share the block lookup and the patch
correction, which are `#[inline(always)]` so the one-off read costs what
it did; out of line they cost it about 6% more instructions.
Instructions per read under callgrind, 256K u32 values:
one-off, no patches 886 -> 887
one-off, patched 4,131 -> 4,130
repeated, no patches 947 -> 797 (-16%)
repeated, patched 4,192 -> 3,959 (-6%)
The repeated case reads a different chunk each time, so every read is
a lane read. The wall-time probe and search benchmarks do few lane reads
per probe and move by at most 2%, within noise.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
It searched 256 chunks 64 times per iteration, about 180 us, far larger than the other Delta benchmarks, which keep to a few chunks so each stays under a millisecond in simulation. It now searches the `SIMULATION_BYTES` array the filter and take benchmarks use, 8 chunks of `u32`, 16 times. Each search still crosses several chunks before it narrows to one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Summary
A Delta point read decoded the whole 1024-value chunk holding the row in order to return one value. That is expensive wherever something searches or slices sorted Delta-encoded indices.
Compressing StatPopGen with editions disabled showed the cost. Element validity compresses to Sparse with Delta-encoded patch indices. Every slice of it binary-searches those indices, and each step of the search decoded a chunk.
Delta transposes each chunk into lanes and encodes every lane against its own base. A value therefore depends only on its lane's base and its lane's deltas up to its row: 8 to 64 deltas, depending on integer width. This PR reads only those, using the single-element undelta that spiraldb/fastlanes#226 added and fastlanes 0.8.1 released.
It is independent of #11452 and #11453, and is based on
develop.Changes
fastlanesgoes from 0.8.0 to 0.8.1.RLE::decode_unchecked's index type to be'static(from Decode RLE with SIMD shuffles and decode Delta-encoded RLE indices in one pass spiraldb/fastlanes#222).decode_chunk_checkedinrle_decompress.rspasses its index type through, so it takes the same bound; the index types are primitive integers.delta/probe.rs: when the deltas are plain or bit-packed on the host, a point read:fastlanes::untranspose, and its lane withfastlanes::delta_lane_row;undelta_singlefor plain deltas, orundelta_pack_singlefor bit-packed ones. Global and per-block bit widths are both supported.u8,u16,u32,u64,i32andi64; plain, globally bit-packed, per-block bit-packed and patched deltas; and bit-packed bases. The test also asserts that these layouts take the lane path.benches/delta_decode.rs: asearch_sortedbenchmark that runs 64 binary searches over a sorted 256-chunk Delta array.Vortex builds
fastlaneswithout its opt-inunrollfeature, so it gets the small single-element undelta. Withunroll, fastlanes compiles an unrolled copy for each width, which runs up to about 2.9× fewer instructions per read for wide deltas but adds about 170 KiB of machine code.Performance
delta_decodebenchmarks, median:developprobe_once(8 one-off reads)search_sorted(64 searches)probe_repeated(8 reads, 2 chunks)StatPopGen's first 10k variants, compressed with editions disabled so Delta is on. These numbers are from an earlier version of this PR that did its own lane arithmetic (median of 3 runs, byte-identical output):
developI have not rerun StatPopGen on the version that uses fastlanes 0.8.1. Its point reads are faster in the benchmarks above.
Checks run
cargo nextest run -p vortex-fastlanes --release: 1032 passed.cargo test --doc -p vortex-fastlanes.cargo clippy -p vortex-fastlanes --all-targets --all-features -- -D warnings.cargo check -p vortex.cargo +nightly-2026-09-10 fmt.Not run:
typos, which is not installed here.vortex-cuda, which also depend onfastlanes; CI covers them.vortex-cudauses onlyfastlanes::FastLanes, which 0.8.1 leaves unchanged.🤖 Generated with Claude Code
https://claude.ai/code/session_01UeniBA4uGoC1yCEQgJdM3J