Repository navigation
Count each run's selected rows with one pass over the RunEnd filter mask - #11221
Merged
Merged
Conversation
`filter_run_end_primitive` counted every run's range of the mask from scratch. On columns with short runs, such as TPC-DS `ss_ticket_number`, that is one popcount call per run, and it took most of the extra CPU of DataFusion scans that apply dynamic filters (q73 SF1: 2.5 s of 2.8 s extra over 150 iterations). Run ends increase, so walk the mask's words once with a running rank instead. `run_end_filter` bench, median before -> after: 4K rows run 16: 1.28 us -> 0.52 us; 16K rows run 16: 4.75 us -> 1.81 us; 64K rows run 4: 65.0 us -> 24.0 us; 64K rows run 256: 1.79 us -> 0.86 us. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
robert3005
approved these changes
Nov 4, 2026
Merging this PR will improve performance by 79.21%1 warning
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
filter_run_end_primitivecalledmask.count_range(run_start, run_end)for every run. On default x86_64 builds, which have no staticpopcnt, each of those calls goes throughfearless_simdruntime dispatch, even when the range is only a few bits long. On columns with short runs, such as TPC-DSss_ticket_number, that per-run overhead was most of the extra CPU in DataFusion scans that apply dynamic filters: 2.5 s of the 2.8 s extra over 150 iterations of q73 at SF1.Split out of #11215.
Changes
MaskRank, which counts the set bits before a position for positions that never decrease, reading each 64-bit mask word once. It builds on Arrow'sBitChunks, which already handles masks that start at a bit offset and zeroes the bits past the end in the final partial word.lengthkeeps them non-decreasing. Each run's selected count is the difference between successive ranks, so the whole mask is walked once instead of each run being counted from scratch.count_rangefor runs of 1–3 rows. The cases cover mask offsets of 0, 3, 5, 61 and 7, a mask ending inside a word, and a mask ending exactly on a word boundary.run_end_filterbench, median before → after:Note: without static
popcnt,u64::count_onescompiles to a software popcount, so for very long runs (thousands of rows per run) the old SIMD range count could be slightly faster per word. The absolute cost is negligible in that case.Generated by Claude Code