Skip to content

Experiment with explicit aggregate input ownership - #10179

Closed
connortsui20 wants to merge 6 commits into
ct/aggregate-array-statsfrom
ct/aggregate-input-owner
Closed

connortsui20 wants to merge 6 commits into
ct/aggregate-array-statsfrom
ct/aggregate-input-owner

Conversation

@connortsui20

@connortsui20 connortsui20 commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Archived ownership prototype while the experiment is simplified. The original implementation and benchmark evidence remain preserved.

Original PR description

Progress towards #10175

Summary

Repeated computation needs an owner that ties retained information to the input that produced it. This experiment introduces that lifetime explicitly so kernels can reuse aggregate results and producer guarantees while cache ownership remains an open question.

Depends on #10178. Array caching remains available for comparison.

Changes

ArrayInput owns an immutable array and aggregate store. Scoped execution supplies its facts to existing cast and take kernels, while producers and stable selections establish or transfer eligible bounds and ordering. Private constructor proofs remain separate from generic aggregate results.

Typed retention keeps rich aggregate states available without scalar conversion, which the compressor experiment builds on. Array, input, and disabled modes exercise the same compute paths. Both stores remain allocated, so lookup bypass alone does not establish a memory saving.

API Changes

Adds the experimental input owner, eager execution methods, and scoped aggregate lookup. The owner has no encoding ID or file representation.

@connortsui20 connortsui20 added the changelog/feature A new feature label Oct 1, 2026
@codspeed

codspeed Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 21.11%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 2 improved benchmarks
❌ 31 regressed benchmarks
✅ 2064 untouched benchmarks
🆕 192 new benchmarks
⏩ 503 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation null_count_run_end[(10000, 1024, 0.01)] 6.4 µs 13.4 µs -52.61%
❌ Simulation null_count_run_end[(32000, 256, 0.01)] 6.4 µs 13.4 µs -52.61%
❌ Simulation null_count_run_end[(10000, 256, 0.01)] 6.4 µs 13.4 µs -52.21%
❌ Simulation null_count_run_end[(32000, 1024, 0.01)] 6.4 µs 13.4 µs -52.01%
❌ Simulation sparse_null_count 62.6 µs 99.8 µs -37.3%
❌ Simulation bitpacked_compress_u32 36.7 µs 58.2 µs -36.9%
❌ WallTime mul_u64_nonnull_neon 28.9 µs 40 µs -27.85%
❌ Simulation for_compress_i32 95.5 µs 130 µs -26.56%
❌ Simulation new_raw_prim_test_between[i32, 2048] 67.6 µs 91.3 µs -25.94%
❌ Simulation compress[(1000, 256)] 98.1 µs 123 µs -20.2%
❌ Simulation compress[(1000, 16)] 104.4 µs 127.3 µs -17.95%
❌ Simulation bitpacked_decompress_u32 51.1 µs 62.1 µs -17.73%
❌ Simulation compress[(1000, 4)] 118 µs 140.4 µs -16%
❌ Simulation sequence_decompress_u32 35.8 µs 42.6 µs -15.93%
❌ WallTime mul_i64_nonnull_neon 32.8 µs 39 µs -15.9%
❌ WallTime multiply_shapes_neon[(32768, PerRowPerRow)] 32.8 µs 39 µs -15.85%
❌ Simulation compress[(4000, 256)] 131 µs 154.8 µs -15.4%
❌ Simulation canonicalize_sparse_list[(1024, 17, 8)] 459.2 µs 539.1 µs -14.82%
❌ Simulation canonicalize_sparse_list[(512, 7, 4)] 534.3 µs 622.6 µs -14.17%
❌ Simulation compress[(4000, 16)] 147.8 µs 171 µs -13.57%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/aggregate-input-owner (34b84b9) with ct/aggregate-array-stats (142bbbc)2

Open in CodSpeed

Footnotes

  1. 503 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on ct/aggregate-array-stats (f69c359) during the generation of this report, so e7865fd was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@connortsui20
connortsui20 force-pushed the ct/aggregate-input-owner branch from b732851 to 7b210bd Compare October 1, 2026 16:01
@connortsui20
connortsui20 added this pull request to stack #10181 October 1, 2026 16:05
@connortsui20
connortsui20 force-pushed the ct/aggregate-input-owner branch 2 times, most recently from 7d17dc1 to 6de2d57 Compare October 1, 2026 16:18
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 force-pushed the ct/aggregate-input-owner branch from 6de2d57 to 34b84b9 Compare October 1, 2026 18:07
@connortsui20
connortsui20 removed this pull request from stack #10181 October 5, 2026 10:43
@connortsui20
connortsui20 deleted the ct/aggregate-input-owner branch October 5, 2026 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant