Repository navigation
Run SQL benchmarks with plan v2 - #9341
joseph-isaacs wants to merge 2 commits into
Conversation
ca0bb6f to
cecbc8f
Compare
Merging this PR will regress 3 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | chunked_constant_i32_append_to_builder[(1000, 10)] |
66.9 µs | 90.9 µs | -26.38% |
| ❌ | Simulation | from_iter_bit_buffer[1024] |
12.6 µs | 14.3 µs | -11.98% |
| ❌ | WallTime | words_gather_scalar[65536] |
8.3 µs | 9.4 µs | -11.89% |
| ⚡ | Simulation | zstd_compress_u32 |
395,371.6 µs | 186.7 µs | ×2,100 |
| ⚡ | Simulation | list_sum_nullable_elements_large |
977,265.1 µs | 720.7 µs | ×1,400 |
| ⚡ | Simulation | listview_sum_large |
224,743.3 µs | 247.7 µs | ×910 |
| ⚡ | Simulation | zstd_compress_string |
265,068.9 µs | 321.5 µs | ×820 |
| ⚡ | Simulation | list_sum_large |
252,885.9 µs | 317.6 µs | ×800 |
| ⚡ | Simulation | fsl_sum_large |
134,419.8 µs | 186.4 µs | ×720 |
| ⚡ | Simulation | zstd_decompress_string |
27,942.8 µs | 135.4 µs | ×210 |
| ⚡ | Simulation | zstd_decompress_u32 |
11,634.9 µs | 81.3 µs | ×140 |
| ⚡ | Simulation | eq_canonicalize_low_match |
28,798.7 µs | 669.9 µs | ×43 |
| ⚡ | Simulation | eq_canonicalize_high_match |
29,088.2 µs | 677 µs | ×43 |
| ⚡ | Simulation | pcodec_compress_f64 |
36,549.5 µs | 870.5 µs | ×42 |
| ⚡ | Simulation | fsst_compress_string |
22,019.3 µs | 526.2 µs | ×42 |
| ⚡ | Simulation | list_length_large |
22,233.5 µs | 537.6 µs | ×41 |
| ⚡ | Simulation | fsst_contains[urls] |
21,002.4 µs | 545.4 µs | ×39 |
| ⚡ | Simulation | listview_length_large |
13,493.3 µs | 353.1 µs | ×38 |
| ⚡ | Simulation | decompress[dict_fsst_varbin_string] |
22,229.2 µs | 625.1 µs | ×36 |
| ⚡ | Simulation | dict_compress_string |
15,928.1 µs | 453.5 µs | ×35 |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ji/vx-plan-ci (4dc8138) with ji/vx-plan-perf (703fdea)
Footnotes
-
442 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
cecbc8f to
ec64cba
Compare
Codecov Report❌ Patch coverage is ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ec64cba to
375dc4e
Compare
3da0336 to
21ac333
Compare
21ac333 to
fe0af76
Compare
fe0af76 to
e77e90b
Compare
e77e90b to
60b67cb
Compare
60b67cb to
07e930c
Compare
dace718 to
a6f3a50
Compare
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
a6f3a50 to
4dc8138
Compare
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
|
This PR was closed because it has been inactive for 7 days since being marked as stale. |
Summary
VORTEX_USE_PLAN_V2=1switch to the DataFusion Vortex file openerEvalexpressions throughZoned, retaining the zones child around the transformed data planFilterMode::{Parallel, Adaptive}to plan-v2: parallel keeps the whole predicate plan, while adaptive chains top-level conjunction plans using v1's selectivity trackingLayoutReaderas the default when the environment variable is unsetWhy
This stacked PR makes the plan-v2 scan path runnable in CI and lets us compare parallel predicate execution with adaptive mask chaining. Every file scan calls
vortex_scan_v2::ScanBuilder::try_new, so benchmark timings include planning and do not use cached or precomputed physical plans.Q19 local results
TPC-H SF=1, DataFusion + Vortex, 100 iterations, file repartitioning disabled:
One-run lineitem scan metrics show the tradeoff: adaptive reduced reads from 56.11 MB / 50 requests to 51.51 MB / 44 requests, but increased scan time from 7.82 ms to 10.13 ms because the mask dependencies serialize the three conjunct plans. Parallel therefore remains the default. The DataFusion adapter accepts
VORTEX_PLAN_V2_FILTER_MODE=adaptivefor direct comparison.Scope and limitations
The DataFusion switch is intentionally experimental. The plan-v2 adapter currently rejects
VortexAccessPlanand partial file ranges; the SQL workflow disables DataFusion file-scan repartitioning so benchmark scans receive whole files.Checks
cargo +nightly fmt --allyamllint --strict -c .yamllint.yaml .github/workflows/sql-bench-matrix.ymlcargo test -p vortex-layoutcargo test -p vortex-scan-v2cargo check -p vortex-datafusionRUSTC_WRAPPER= cargo build -p datafusion-bench --profile release_debug --features unstable_encodingsRUSTC_WRAPPER= VORTEX_USE_PLAN_V2=1 cargo test -p vortex-datafusion persistent::opener::tests::test_open -- --exactRUSTC_WRAPPER= cargo clippy --all-targets --all-features -- -D warningsgit diff --check