Repository navigation
Execute zoned plan pruning - #9244
joseph-isaacs wants to merge 1 commit into
Conversation
Merging this PR will improve performance by ×83
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | struct_mixed_vortex |
65,287.3 µs | 580.2 µs | ×110 |
| ⚡ | Simulation | struct_mixed_arrow_row |
49,050.1 µs | 507.5 µs | ×97 |
| ⚡ | Simulation | utf8_arrow_row |
20,523.3 µs | 219.3 µs | ×94 |
| ⚡ | Simulation | utf8_vortex |
20,981 µs | 242.6 µs | ×86 |
| ⚡ | Simulation | primitive_i64_arrow_row |
6,234.2 µs | 75.7 µs | ×82 |
| ⚡ | Simulation | primitive_i64_vortex |
4,036.2 µs | 87.2 µs | ×46 |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing vortex-plan-zoned-pruning (5866912) with vortex-plan-scan-v2 (0fe3ac1)
Footnotes
-
1996 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
7d7bdf1 to
57d4c88
Compare
57d4c88 to
1bdf135
Compare
1bdf135 to
dd57303
Compare
dd57303 to
de37c0f
Compare
de37c0f to
1673868
Compare
3eb5fb2 to
8294f49
Compare
59167d7 to
ff445e9
Compare
ff445e9 to
61f20f3
Compare
61f20f3 to
5c193ae
Compare
5c193ae to
dd71e39
Compare
1ded40b to
bb8da75
Compare
f198ffb to
863ed00
Compare
ef9eabf to
fd4f91c
Compare
fd4f91c to
eb8c6ed
Compare
eb8c6ed to
3bd86f6
Compare
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
3bd86f6 to
5866912
Compare
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
|
This PR was closed because it has been inactive for 7 days since being marked as stale. |
Summary
ZonedPlanzones childStack and ownership
This is based on #9240. Zoned planning and the static
(Eval, Zoned)rewrite live in #9187; generic physical-plan execution lives in #9240; this PR only materializes and executes zone-backed pruning.Performance follow-up
The current implementation prioritizes correctness and keeps pruning inside the general array-plan execution contract. On the 100-shard ClickBench scan, pruning eliminated 51.8% of rows and fully skipped 264 of 1,100 splits, but prepared v2 execution was 44.41 ms with pruning versus 30.69 ms without it.
A follow-up should make pruning mask-native and preserve encoded filter evaluation before combining the pruning mask. That performance work remains separate from this execution PR.
Validation
cargo test -p vortex-layout -p vortex-scan-v2— 250 layout tests and 8 scan-v2 tests passed at this stack layercargo test -p vortex-layout -p vortex-scan-v2 -p vortex-datafusionRUSTC_WRAPPER= VORTEX_USE_PLAN_V2=1 cargo test -p vortex-datafusion persistent::opener::tests::test_open -- --exactcargo +nightly fmt --all -- --checkcargo clippy --all-targets --all-features -- -D warningsgit diff --check origin/develop...HEAD