Skip to content

Benchmark sparse-list canonicalization with a null fill - #11175

Merged
gatesn merged 3 commits into
developfrom
ngates/bench-sparse-list-null-fill
Nov 3, 2026
Merged

gatesn merged 3 commits into
developfrom
ngates/bench-sparse-list-null-fill

Conversation

@gatesn

@gatesn gatesn commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Adds null-fill cases to the existing vortex-sparse sparse-list canonicalization benchmark, as requested in the #11173 discussion.

The new canonicalize_sparse_list_null_fill cases use the same (len, patch_stride, list_size) inputs as the existing cases: (512, 7, 4) and (1024, 17, 8). Present patch lists are valid; rows between patches are null. Fixture construction and execution-context creation stay outside the timed canonicalization. Existing nonempty-fill and fixed-size-list benchmark names and inputs are preserved.

This is a benchmark-only PR directly against develop, with none of #11173's implementation changes. Land it first to establish a CodSpeed baseline, then rebase #11173 and compare the new cases. The existing CodSpeed configuration already includes the sparse_canonical target.

Validation

  • cargo clippy -p vortex-sparse --bench sparse_canonical --all-features -- -D warnings
  • cargo bench -p vortex-sparse --bench sparse_canonical --profile dev -- --test — all six cases passed; this was a smoke run, not an optimized timing comparison.
  • cargo +nightly-2026-09-10 fmt -p vortex-sparse --check
  • git diff --check

For an optimized local measurement:

cargo bench -p vortex-sparse --bench sparse_canonical -- canonicalize_sparse_list_null_fill

Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn added the changelog/chore A trivial change label Nov 3, 2026 — with ChatGPT Codex Connector
@gatesn
gatesn marked this pull request as ready for review November 3, 2026 19:36
@gatesn
gatesn enabled auto-merge (squash) November 3, 2026 19:36
Signed-off-by: Nicholas Gates <nick@nickgates.com>
This reverts commit 47927e1.

Signed-off-by: Nicholas Gates <nick@nickgates.com>
@codspeed

codspeed Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 11.61%

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚡ 1 improved benchmark
✅ 1881 untouched benchmarks
🆕 2 new benchmarks
⏩ 469 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Walltime lt_i64_nullable_avx512 8.2 µs 7.4 µs +11.61%
🆕 CPU Simulation canonicalize_sparse_list_null_fill[(1024, 17, 8)] N/A 432.8 µs N/A
🆕 CPU Simulation canonicalize_sparse_list_null_fill[(512, 7, 4)] N/A 436.4 µs N/A

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ngates/bench-sparse-list-null-fill (55b020e) with develop (8bada73)

Open in CodSpeed

Footnotes

  1. 469 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@gatesn
gatesn merged commit 0c84f00 into develop Nov 3, 2026
94 of 95 checks passed
@gatesn
gatesn deleted the ngates/bench-sparse-list-null-fill branch November 3, 2026 20:05
gatesn added a commit that referenced this pull request Nov 3, 2026
Rebased onto `develop` at `8896ab27d0`, including the benchmark baseline
merged in #11175.

**Validation:** all 54 checks pass, including CodSpeed Performance
Analysis. Only the human-review policy is pending.

This optimizes read-time reconstruction of sparse lists. It reuses their
existing elements and builds offsets directly, instead of rebuilding the
lists through a builder.

For example, suppose only two rows contain lists:

```text
row:       0      1     2      3      4
value:   [a,b]   null  null  [c,d]   null
```

The elements are already stored compactly as `[a,b,c,d]`. To represent
all five rows as a native List, we only need offsets describing which
elements belong to each row:

```text
elements: [a, b, c, d]          (reuse the existing array)
offsets:  [0, 2, 2, 2, 4, 4]   (one boundary per row, plus the final boundary)
```

Previously, sparse decoding built the output by appending patch-list
slices and fill gaps through a list builder. When the elements are a
lazy gather, those appends can repeatedly execute the same gather
indices.

The new main path keeps the compact elements and constructs the offsets
directly. Empty or null fill rows consume no elements, so their offsets
repeat. Store those repetitions as runs:

```text
patch positions:  [0, 3]
compact offsets:  [0, 2, 4]
offset run ends:  [1, 4, 6]
offset values:    [0, 2, 4]
decoded offsets:  [0, 2, 2, 2, 4, 4]
```

This avoids rebuilding elements and keeps offsets compressed until a
consumer needs them expanded. Arrow's native List export still
materializes dense offsets. The optimization changes read-time
reconstruction, not file compression or size.

## Implementation

The main `empty_fill` function now follows the native-list path
directly: reuse the compact patch elements, construct run-end offsets,
and retain validity. `can_use_native_list` isolates the representability
checks. `empty_fill_as_views` separately handles duplicate patch
positions (last patch wins) and narrow view offsets whose terminal
offset would overflow; it scatters independent offsets/sizes while
sharing the elements.

Nonempty fills and batches with at most one valid patch keep the
existing builder path: there is no repeated element gather to avoid in
the latter case. Duplicate positions still use the metadata path to
preserve last-patch-wins semantics. Regression tests cover null/empty
fills, repeated positions, genuine null patches, element-buffer sharing,
the physical offset representation, and terminal-offset overflow. The
sparse struct writer remains separate.

## Benchmark comparison

### CodSpeed CPU simulation

[Final
report](#11173 (comment)),
comparing PR `c981287` with develop `1012d0c` (which includes the #11175
baseline):

| Benchmark | Baseline µs | PR µs | Baseline / PR |
| --- | ---: | ---: | ---: |
| Null-fill list `(512, 7, 4)` | 423.4 | 200.3 | **2.11×** |
| Null-fill list `(1024, 17, 8)` | 421.9 | 227.5 | **1.85×** |

CodSpeed reports two improvements and no regressions. Both
compressed-child union-take cases were measured in this run and no
longer flag regressions. Keeping the builder for a single valid list
avoids offset setup when there is no repeated gather to eliminate.

These results measure CPU simulation. The local wall-clock measurements
below use a different method and platform.

### Local wall-clock

Local macOS arm64 measurement, Rust 1.99.1, `release_debug` with the
repository's default codegen settings. Baseline `8896ab27d0` versus this
PR at `c981287d20`; identical benchmark source from #11175. Five
alternating/reversed baseline/candidate pairs, at least 1,000 samples
per case and a minimum of 0.25 seconds per case, one thread. Values are
medians of the five per-run medians.

| Case | Rows / patch stride / list size | Baseline µs | PR µs | Speedup
|
| --- | --- | ---: | ---: | ---: |
| Null-fill list | 512 / 7 / 4 | 11.240 | 2.666 | **4.22×** |
| Null-fill list | 1024 / 17 / 8 | 10.620 | 2.874 | **3.70×** |
| Nonempty-fill list control | 512 / 7 / 4 | 10.740 | 10.700 | 1.00× |
| Nonempty-fill list control | 1024 / 17 / 8 | 9.040 | 9.082 | 1.00× |
| Fixed-size-list control | 512 / 7 / 4 | 11.200 | 11.160 | 1.00× |
| Fixed-size-list control | 1024 / 17 / 8 | 10.870 | 10.830 | 1.00× |

The null-fill cases improve substantially; this microbenchmark's control
changes are within 1%. With the single-valid-patch guard, five
alternating local union-take pairs measure 6.040 → 5.874 µs and 7.291 →
7.207 µs for the compressed-child cases; dense-child controls are
unchanged. The final CodSpeed report also flags no union-take
regressions. This measures top-level canonicalization with primitive
patch elements, not Arrow export or the separate lazy-gather workload.

Both binaries were built in the same dedicated checkout, preserving the
baseline executable before checking out the candidate. Each was run
with:

```sh
<benchmark-binary> --bench --color never --threads 1 --sample-count 1000 --min-time 0.25
```

Raw local results and the comparison script are saved at
`/private/tmp/vortex-11173-benchmark-results`.

## Validation

- `cargo clippy -p vortex-sparse --all-targets --all-features -- -D
warnings`
- `cargo nextest run -p vortex-sparse --lib --all-features` — 129
passed, including single-valid-patch and duplicate-position coverage.
- `cargo +nightly-2026-09-10 fmt -p vortex-sparse --check`
- `git diff --check`

Targets `develop` directly. Merge before #11174: that
dictionary/sparse-code rewrite exposes the expensive list path addressed
here. #11172 independently avoids rebuilding null-only constant list
batches.

---------

Signed-off-by: Nicholas Gates <nick@nickgates.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants