Repository navigation
Compute integer statistics in one hand-written pass per need - #10346
Conversation
Merging this PR will improve performance by ×22 warnings
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
2631c40 to
941b57d
Compare
Codecov Report❌ Patch coverage is
☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
477507c to
0723534
Compare
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
Read the validity a word per chunk of 64 values instead of slicing a bit buffer per chunk, and fill each null with the valid value before it, so that the bounds and the runs take the vectorized kernel of non-null chunks. Distinct counting visits only the set bits. Compute min/max with lane accumulators in the same pass as the runs, or as the distinct counts for bytes, instead of a separate min_max aggregate over each valid range. Write the bounds and the null count back to the array statistics, as the aggregate did, and reuse cached bounds. Wider types counting distinct values still compute the bounds first, in their own pass, to choose a counter. Choose the distinct counter once and monomorphize the pass on it, and keep the run state in locals so that count updates cannot alias it. Skip counting when the bounds show every valid value is the same. Fold the lanes behind a reference in a function of their own: as locals of the large caller, the vectorizer left them scalar. Choose the lane count per instruction set: with AVX2, 32-bit bounds fold over a whole chunk and 64-bit bounds over 8 lanes. Tidy the integer stats scan: Give the lane bounds their own constructor and reduction, convert the dense counter's bounds to i128 once, name the empty distinct info, and make the chunk accumulator's control flow explicit instead of returning early. Callgrind shows no regression on any type with or without AVX2; u8 and u16 distinct counting runs about 6% fewer instructions. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Signed-off-by: Robert Kruszewski <github@robertk.io>
…lone Without AVX2, 32-bit bounds fold over 32 lanes instead of 16. On an Apple M4, 16 lanes made the non-null stats of u32 twice as slow as develop. Where the runs do not need the nulls filled, which is the bounds pass before distinct counting and the counting pass of bytes, values of at most 32 bits read each null as the identity of each bound instead of filling it. The select is a mask of all ones or all zeros: as an `if`, LLVM branches on each validity bit and leaves the lanes scalar. 64-bit values keep the fill, which is cheaper with their few lanes. integer_stats, 64Ki u32, Apple M4, median µs: | | develop | before | after | |----------------|--------:|-------:|------:| | non_null | 7.4 | 16.1 | 6.5 | | nullable | 48.7 | 22.6 | 13.7 | | all_non_null | 63-81 | 72-81 | 72-79 | | all_nullable | 99-145 | 58-73 | 46-49 | Signed-off-by: Robert Kruszewski <github@robertk.io>
0d9fdeb to
e234151
Compare
…sk_word Add `for_each_mask_word` and `try_for_each_mask_word` to `vortex-compute`. They walk a `BitBufferView` one `u64` word at a time through `unaligned_chunks`, so the 8-byte aligned body is read with no reshift for each word. Use the walk in these places instead of a local copy: - the masked lane kernels `map_masked_into`, `try_map_masked_into` and `try_map_masked_in_place`, which now accept `impl Into<BitBufferView>` - the filter word walk for a mask bitmap - the primitive and ListView zip kernels - the FoR encoder for chunks with mixed validity, which no longer copies the validity words into a `Vec` Compute the float statistics over chunks of 64 values. A chunk that is fully valid has a branch-free count of its runs, and a mixed chunk gathers its valid values first. The integer statistics are not changed. #10346 rewrote them on develop and that replaces the changes this branch made to them. Signed-off-by: Robert Kruszewski <github@robertk.io>
Summary
Computes the integer compression statistics in one hand-written pass per need. Only
vortex-compressor/src/stats/integer.rschanges, and every statistic has the same value as before.u64per chunk of 64 values. For the runs, nulls are filled with the previous valid value. When only the bounds are needed, values of at most 32 bits read nulls as the identity instead, with a branch-free select.min_maxaggregate. The bounds and the null count are cached on the array, and bounds already cached there are reused.The tests compare min, max, runs and distinct counts against a naive reference for
u8,i16,u32,i64andu64.Benchmark
integer_stats, 64Kiu32, median µs:non_nullnullableall_non_nullall_nullableOn M4,
u8,u16andu64with nulls are 2–4× faster. Without nulls, distinct counting is slower than develop (u840 → 56 µs,u6449 → 68 µs), and so areu64statistics without distinct counting (17 → 24 µs).integer_statscovers onlyu32.🤖 Generated with Claude Code