Skip to content

Grouped SumV2 kernel for run-end list elements - #11107

Draft
mhk197 wants to merge 1 commit into
mk/grouped-sum-sparsefrom
mk/grouped-sum-run-end
Draft

mhk197 wants to merge 1 commit into
mk/grouped-sum-sparsefrom
mk/grouped-sum-run-end

Conversation

@mhk197

@mhk197 mhk197 commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a grouped SumV2 kernel for run-end elements. Each list sums its runs with sum_range instead of decoding. With the whole stack, statpopgen (DataFusion, list layout, with #10854) runs q3/q4 at 0.35× and q6 at 0.28× (geomean 0.687×).

Changes

  • RunEndGroupedSumV2Kernel, registered in AggregateFnSession; bool values sum as u8.
  • grouped_partials becomes pub(crate) for the kernel.

Benchmarks

Time vs the PR below, by average list length.

bench 8 256 4096
run_end_bool_sum 2.25× 1.00× 0.66×
chunked_run_end_bool_sum 1.51× 0.97× 0.74×

Testing

  • grouped_kernels_match_per_group_sums: run-end, run-end bool, sliced and overflow cases.
  • cargo clippy -p vortex-array --all-targets --all-features -- -D warnings and the vortex-array sum tests pass.

Follow-up

  • Lists of 8 are slower than decoding and counting bits; the kernel could decline short lists.

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant