Skip to content

Benchmark integer statistics with and without nulls on CodSpeed - #10341

Merged
joseph-isaacs merged 1 commit into
developfrom
ji/int-stats-bench
Oct 6, 2026
Merged

joseph-isaacs merged 1 commit into
developfrom
ji/int-stats-bench

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

No benchmark tracks the integer compression statistics in CI. The vortex-compressor benchmarks are all #[cfg(not(codspeed))], the package is not in any CodSpeed shard, and stats_calc only builds non-nullable arrays. This adds a small benchmark CodSpeed runs, so the follow-up PR that speeds up the nullable path has a baseline.

Changes

  • vortex-compressor/benches/integer_stats.rs: two benchmarks of IntegerStats::generate_opts over 64Ki u32 values in short runs (up to 8, 1024 distinct):

    • integer_stats_non_null
    • integer_stats_nullable: the same values with one in ten null at random, so almost every chunk of 64 values has a null.

    Each iteration builds a fresh array, so the min/max cached on the array by a previous iteration is not reused.

  • .github/workflows/codspeed.yml: add vortex-compressor to shard 6. Its other benchmarks stay excluded from CodSpeed.

The compressor benchmarks are excluded from CodSpeed, and none covers
nullable arrays. Add one benchmark of integer statistics over a
non-null array, and one over the same array with one value in ten null,
and run the compressor benchmarks in a CodSpeed shard.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018J6qahqrVFb4UCPkwUzY11
IntegerStats::generate_opts(
array,
GenerateStatsOptions {
count_distinct_values: false,

@connortsui20 connortsui20 Oct 6, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably want this a true since we ALWAYS have this set to true in the compressor

@connortsui20 connortsui20 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably should also add the count distinct value bench later

@joseph-isaacs
joseph-isaacs merged commit 19a8a9c into develop Oct 6, 2026
126 of 128 checks passed
@joseph-isaacs
joseph-isaacs deleted the ji/int-stats-bench branch October 6, 2026 16:03
connortsui20 pushed a commit that referenced this pull request Oct 7, 2026
## Summary

Stacked on #10341, which adds the benchmarks.

Integer statistics with nulls sliced the validity `BitBuffer` for every
chunk of 64 values to count its set bits, then tested each value's bit
with a branch. With nulls spread out, almost no chunk is fully valid
(0.9⁶⁴ ≈ 0.1% at 10% nulls), so nearly every chunk took that slow path.

## Changes

In `vortex-compressor/src/stats/integer.rs`:

- Read the validity one `u64` word per chunk instead of slicing a bit
buffer. A word of zeros is skipped, and a full word takes the non-null
loop as before.
- Without distinct values, fill each null with the valid value before
it. A filled null adds no run, so the chunk takes the same branch-free
run count as a non-null chunk.
- With distinct values, push only the set bits, since a filled null
would count as an occurrence.
- The trailing values are padded into a last chunk whose padding is
null. This replaces the separate per-bit remainder loop.

Min/max is unchanged: it is still computed by the `min_max` aggregate
before this loop.

`test_matches_naive_reference` now also covers statistics without
distinct values and chunks that are entirely null. Filling the nulls
with their own values instead makes 38 of its cases fail.

---------

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/skip Do not list PR in the changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants