Skip to content

feat(scan): planner-driven V2 executor for projection scans behind one fork - #10530

Closed
joseph-isaacs wants to merge 8 commits into
developfrom
claude/planner-decode-slice-migration-xppbf1
Closed

joseph-isaacs wants to merge 8 commits into
developfrom
claude/planner-decode-slice-migration-xppbf1

Conversation

@joseph-isaacs

Copy link
Copy Markdown
Contributor

Summary

First mergeable slice of the V1 to V2 scan migration, following the handoff in vortex-scan/design/planner-decode-slice.md (#10464 / #10466 trimmed down). Projection-only scans can run on develop's pipeline Scan, driven by the vortex-scan planning protocol: every segment read is an IoRequest served from the file's SegmentSource.

V1 and V2 fork in one place only. The source commit put if v2::enabled() && v2::supports_scan(..) { v2::into_stream(..) } else { b.into_stream() } at every caller. Here, ScanBuilder's lazy prepare step is the only fork:

fn prepare_stream(self) -> VortexResult<BoxStream<'static, VortexResult<A>>> {
    match v2::select(self)? {
        Executor::V1(builder) => builder.prepare()?.execute_stream(None),
        Executor::V2(scan) => scan.execute_stream(),
    }
}

v2::select owns the gate: VORTEX_SCAN_V2=1, the builder carries a ScanFile, there is no filter or limit, and the file has no List layouts (lists written as flat leaves, the default, are supported). Each later PR widens this one gate. into_stream, into_iter and the Arrow record-batch stream all go through it, so no caller changes. The decision is logged at debug on target vortex_layout::scan::v2.

Changes

  • vortex-scan: the planning driver (Run / Driver) from feat(scan): planner-driven V2 scans on the physical plan pipelines #10464, with tracing stripped so it does not depend on feat(file): a coalescing IO service for planning-protocol scans #10463, plus its scripted tests.
  • vortex-layout:
    • scan/planning/:
      • ProtocolGraph maps pipeline Turn::Read to Fetch requests.
      • ProjectionPlanner / ProjectionMorsel with prefetch and forget dropped.
    • scan/v2/:
      • the gate (select) and PreparedScan.
      • SegmentScanIo adapts a SegmentSource to IoService.
      • SplitPlan and the chunk-start splitter.
      • Splits are spawned lazily with V1's ordered/unordered buffering and per-thread concurrency.
    • ScanBuilder::with_scan_file.
  • vortex-file: VortexFile::scan_file(), and VortexFile::scan() attaches it.
  • Fixes found while benchmarking:
    • Develop's lower keeps Zoned/LegacyStats plan nodes, so the boundary walk now follows their data child. Before this, every default-written file ran as a single split.
    • Decoded dictionary values are shared across all splits of a scan, as V1's reader does. Per-split caches made HashTags 2.2x and CMSprovider 1.7x slower.
  • Tests (scan/v2/tests.rs):
    • V2 output equals V1 over flat, table and zoned layouts, with row ranges, row indices and a row_idx projection.
    • Filter, limit, no-ScanFile and List-layout scans stay on V1.
    • Unordered scans return every row.
    • Scans split at chunk starts (this test fails without the zoned fix).

Local verification

  • cargo clippy -p vortex-scan -p vortex-layout -p vortex-file --all-targets --all-features -- -D warnings: clean.
  • cargo +nightly-2026-09-10 fmt on those crates.
  • cargo test -p vortex-layout --lib: 530 passed.
  • cargo test -p vortex-scan --lib: 59 passed.
  • compress-bench --formats vortex --ops decompress with VORTEX_SCAN_V2=1: all 15 datasets ran on V2 (15 scan runs on v2, 0 fallbacks).
  • Output parity: a temporary, uncommitted harness wrote every benchmark Parquet file (PBI, taxi, TPC-H SF1 tables, struct-list-of-ints) to Vortex, scanned it with both executors and compared the Arrow output. All 22 files matched.
  • Not run: vortex-file tests, and the DataFusion / DuckDB integrations (they build their own ScanBuilder and stay on V1 until they call with_scan_file).

Local decompression times, 4-core container, 5 iterations, V1 (VORTEX_SCAN_V2=0) vs V2 (=1):

Dataset V1 (ms) V2 (ms) V2 / V1
taxi 72.8 62.5 0.86x
Arade 88.2 89.5 1.01x
Bimbo 467.9 452.4 0.97x
CMSprovider 332.6 342.2 1.03x
Euro2016 87.7 85.9 0.98x
Food 35.6 28.5 0.80x
HashTags 311.4 336.1 1.08x
TPC-H l_comment chunked 119.6 113.3 0.95x
TPC-H l_comment canonical 115.9 111.7 0.96x
wide table cols=100 chunks=1 2.0 1.9 0.96x
wide table cols=1000 chunks=1 18.3 21.1 1.15x
wide table cols=10000 chunks=1 170.5 207.9 1.22x
wide table cols=100 chunks=50 2.1 2.6 1.25x
wide table cols=1000 chunks=50 19.2 18.7 0.97x
wide table cols=10000 chunks=50 211.8 181.8 0.86x

Geomean V2/V1: 1.00x. These are single local runs on a noisy shared container. The design doc's open risk (the full stack regressed on CI's 96-core runners) still needs a CI benchmark run to rule out.

API Changes

All additive. V1 behaviour is unchanged unless VORTEX_SCAN_V2=1 is set:

  • ScanBuilder::with_scan_file
  • VortexFile::scan_file
  • vortex_layout::scan::v2::{ScanFile, SegmentLocation, ENV_VAR, enabled}
  • vortex_scan::planning::driver

🤖 Generated with Claude Code

https://claude.ai/code/session_017cUKVPYqfN7SLcXetK59i6


Generated by Claude Code

Describe how to land the vortex-scan planning driver running develop's
pipeline::Scan decode path, with IO through the planning protocol, as a
small PR stack extracted from 42b29cc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P1oQtzTgSzkvrGDKisoNko
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Claude <noreply@anthropic.com>
…e fork

Port the planning driver (`Run`/`Driver`) into vortex-scan and a
projection-only V2 executor into vortex-layout that runs develop's
pipeline `Scan` through the planning protocol, with every segment read
issued as an `IoRequest` served from the file's `SegmentSource`.

`ScanBuilder`'s lazy prepare step is the single place a streaming scan
picks its executor: `v2::select` returns V1 unless `VORTEX_SCAN_V2=1`,
the builder carries a `ScanFile`, and the scan has no filter, no limit
and no list layout. `VortexFile::scan()` attaches the `ScanFile`; no
caller changes.

Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017cUKVPYqfN7SLcXetK59i6
…ndoff doc

Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017cUKVPYqfN7SLcXetK59i6
Develop's lower keeps Zoned and LegacyStats nodes in the plan, so the
boundary walk stopped at them and every default-written file ran as a
single split. Follow their data child.

Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017cUKVPYqfN7SLcXetK59i6
Each split had its own ValueCache, so a dictionary shared by many chunks
was decoded once per split (HashTags 2.2x, CMSprovider 1.7x slower than
V1). Share one cache per scan, as V1's reader does.

Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017cUKVPYqfN7SLcXetK59i6
@codspeed

codspeed Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 4 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 19 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

ℹ️ 40 benchmarks have no flame graph

The flamegraph could not be generated because we failed to resolve the symbols in the profiling trace. This happens when we fail to harvest information about language frames.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 5 improved benchmarks
❌ 4 regressed benchmarks
✅ 2250 untouched benchmarks
🆕 9 new benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ ⚠️ Simulation test_repeated_scan_scalar_at[int_str_float] 539 µs 628.6 µs -14.24%
❌ ℹ️ Simulation test_scan_filter_to_arrow_array[int_float] 3.5 ms 4.1 ms -13.72%
❌ ℹ️ Simulation test_repeated_scan_filter[int_float] 3.4 ms 3.9 ms -11.17%
❌ ℹ️ Simulation test_repeated_scan_filter[int] 2.5 ms 2.7 ms -10.44%
⚡ ℹ️ Simulation test_repeated_scan_scalar_at[int_float] 486.7 µs 393.3 µs +23.77%
⚡ WallTime bitpack_blocked_compress_avx512 5.6 µs 4.8 µs +18.01%
⚡ WallTime bitpack_blocked_compress_avx2 7.6 µs 6.7 µs +13.08%
⚡ WallTime filter_neon[80] 2.9 µs 2.6 µs +12.48%
⚡ WallTime words_gather_dispatch_avx2[65536] 1.4 µs 1.3 µs +10.27%
🆕 Simulation prepare_and_compile[1] N/A 52.7 µs N/A
🆕 Simulation prepare_and_compile[100] N/A 322.3 µs N/A
🆕 Simulation prepare_and_compile[1000] N/A 2.6 ms N/A
🆕 Simulation prepare[1] N/A 16.3 µs N/A
🆕 Simulation prepare[100] N/A 74 µs N/A
🆕 Simulation prepare[1000] N/A 424 µs N/A
🆕 Simulation scan[1] N/A 96.6 µs N/A
🆕 Simulation scan[100] N/A 1.3 ms N/A
🆕 Simulation scan[1000] N/A 12.3 ms N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/planner-decode-slice-migration-xppbf1 (7b15cfc) with develop (18aebea)

Open in CodSpeed

Reuse V1 split policies, avoid redundant identity-projection planning, and release decoded dictionary values after their consuming splits finish. Reduce pipeline allocation, plan retention, and ready-I/O scheduling overhead.

Validated with repeated V1/V2/Parquet decompression comparisons, output parity across 14 datasets, and 175 focused pipeline and scan tests. Some workloads still retain modest RSS and parallel duplicate-read overhead versus V1.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Measure preparation, compilation and in-memory segment delivery for narrow
and wide pipeline scans in the existing CodSpeed layout shard. Document the
stable identity of scoped dictionary-cache keys and use an explicit Arc
clone in the V2 test so the scan code passes the CI Clippy configurations.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Keep private implementation names as code and remove the redundant v2 link
target so the planner documentation builds with warnings denied.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the action/bench-compress Run only the compression benchmark on this PR label Oct 16, 2026
@github-actions github-actions Bot removed the action/bench-compress Run only the compression benchmark on this PR label Oct 16, 2026
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 7b15cfc compress-bench 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Compression 📖

Commits: PR 7b15cfc7 vs base 18aebeaa


vortex / vortex-file-compressed / ns (0.990x ➖, 1↑ 0↓)
name ns (PR / base / %diff)
compress time/Arade 769504907 / 787435434 / -2.3%
compress time/Bimbo 3567485499 / 3692508802 / -3.4%
compress time/CMSprovider 2517975685 / 2545029989 / -1.1%
compress time/Euro2016 269587480 / 267512769 / +0.8%
compress time/Food 317681583 / 317789581 / -0.0%
compress time/HashTags 463339454 / 463547621 / -0.0%
compress time/TPC-H l_comment canonical 613412613 / 600311849 / +2.2%
compress time/TPC-H l_comment chunked 609218266 / 603742733 / +0.9%
compress time/taxi 390186460 / 387878829 / +0.6%
compress time/wide table cols=100 chunks=1 rows=1000 7072264 / 7082137 / -0.1%
compress time/wide table cols=100 chunks=50 rows=1000 7212911 / 7133163 / +1.1%
compress time/wide table cols=1000 chunks=1 rows=1000 70325125 / 71662532 / -1.9%
compress time/wide table cols=1000 chunks=50 rows=1000 69987129 / 70054687 / -0.1%
compress time/wide table cols=10000 chunks=1 rows=1000 763836076 / 769400585 / -0.7%
compress time/wide table cols=10000 chunks=50 rows=1000 774863908 / 778349156 / -0.4%
decompress time/Arade 8781591 / 9053434 / -3.0%
decompress time/Bimbo 35359005 / 37381017 / -5.4%
decompress time/CMSprovider 27103991 / 27149413 / -0.2%
decompress time/Euro2016 9897031 / 9746141 / +1.5%
decompress time/Food 2310517 / 2908948 / -20.6% 🟢
decompress time/HashTags 42365680 / 43177640 / -1.9%
decompress time/TPC-H l_comment canonical 11303669 / 11031972 / +2.5%
decompress time/TPC-H l_comment chunked 10653348 / 10909365 / -2.3%
decompress time/taxi 5170664 / 5181307 / -0.2%
decompress time/wide table cols=100 chunks=1 rows=1000 1308239 / 1345212 / -2.7%
decompress time/wide table cols=100 chunks=50 rows=1000 1315284 / 1348698 / -2.5%
decompress time/wide table cols=1000 chunks=1 rows=1000 12824635 / 12460271 / +2.9%
decompress time/wide table cols=1000 chunks=50 rows=1000 12825222 / 12409842 / +3.3%
decompress time/wide table cols=10000 chunks=1 rows=1000 136647932 / 133669276 / +2.2%
decompress time/wide table cols=10000 chunks=50 rows=1000 137355454 / 133497275 / +2.9%
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
name bytes (PR / base / %diff)
vortex-file-compressed size/Arade 142213756 / 142213756 / +0.0%
vortex-file-compressed size/Bimbo 458581212 / 458581212 / +0.0%
vortex-file-compressed size/CMSprovider 420794836 / 420794836 / +0.0%
vortex-file-compressed size/Euro2016 164130260 / 164125572 / +0.0%
vortex-file-compressed size/Food 38950752 / 38950752 / +0.0%
vortex-file-compressed size/HashTags 196170516 / 196170516 / +0.0%
vortex-file-compressed size/TPC-H l_comment canonical 178688232 / 178688232 / +0.0%
vortex-file-compressed size/TPC-H l_comment chunked 178688232 / 178688232 / +0.0%
vortex-file-compressed size/taxi 49998092 / 49998092 / +0.0%
vortex-file-compressed size/wide table cols=100 chunks=1 rows=1000 934080 / 934080 / +0.0%
vortex-file-compressed size/wide table cols=100 chunks=50 rows=1000 934080 / 934080 / +0.0%
vortex-file-compressed size/wide table cols=1000 chunks=1 rows=1000 9325680 / 9325680 / +0.0%
vortex-file-compressed size/wide table cols=1000 chunks=50 rows=1000 9325680 / 9325680 / +0.0%
vortex-file-compressed size/wide table cols=10000 chunks=1 rows=1000 93277680 / 93277680 / +0.0%
vortex-file-compressed size/wide table cols=10000 chunks=50 rows=1000 93277680 / 93277680 / +0.0%
vortex / vortex-file-compressed / ratio (0.991x ➖, 1↑ 0↓)
name ratio (PR / base / %diff)
vortex:parquet-zstd ratio compress time/Arade 0.342115326 / 0.350073615 / -2.3%
vortex:parquet-zstd ratio compress time/Bimbo 0.302524597 / 0.311439563 / -2.9%
vortex:parquet-zstd ratio compress time/CMSprovider 0.394447416 / 0.398612965 / -1.0%
vortex:parquet-zstd ratio compress time/Euro2016 0.250094311 / 0.2472584 / +1.1%
vortex:parquet-zstd ratio compress time/Food 0.429958126 / 0.429202695 / +0.2%
vortex:parquet-zstd ratio compress time/HashTags 0.253512545 / 0.25390901 / -0.2%
vortex:parquet-zstd ratio compress time/TPC-H l_comment canonical 0.234269095 / 0.228396024 / +2.6%
vortex:parquet-zstd ratio compress time/TPC-H l_comment chunked 0.231792407 / 0.229279183 / +1.1%
vortex:parquet-zstd ratio compress time/taxi 0.358015924 / 0.356144023 / +0.5%
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=1 rows=1000 1.77853311 / 1.79102265 / -0.7%
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=50 rows=1000 1.82973837 / 1.76216895 / +3.8%
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=1 rows=1000 1.57776844 / 1.61095158 / -2.1%
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=50 rows=1000 1.54628534 / 1.58194927 / -2.3%
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=1 rows=1000 1.63878205 / 1.67738835 / -2.3%
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=50 rows=1000 1.67823147 / 1.72058246 / -2.5%
vortex:parquet-zstd ratio decompress time/Arade 0.015791212 / 0.0162564564 / -2.9%
vortex:parquet-zstd ratio decompress time/Bimbo 0.0220653715 / 0.0233227159 / -5.4%
vortex:parquet-zstd ratio decompress time/CMSprovider 0.0199162221 / 0.0199083036 / +0.0%
vortex:parquet-zstd ratio decompress time/Euro2016 0.0297823382 / 0.0293179881 / +1.6%
vortex:parquet-zstd ratio decompress time/Food 0.0136500126 / 0.017115439 / -20.2% 🟢
vortex:parquet-zstd ratio decompress time/HashTags 0.0822907703 / 0.0835751162 / -1.5%
vortex:parquet-zstd ratio decompress time/TPC-H l_comment canonical 0.0227095618 / 0.0221928205 / +2.3%
vortex:parquet-zstd ratio decompress time/TPC-H l_comment chunked 0.0213746458 / 0.0219569447 / -2.7%
vortex:parquet-zstd ratio decompress time/taxi 0.0237704327 / 0.0238150044 / -0.2%
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=1 rows=1000 0.643179341 / 0.653945343 / -1.6%
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=50 rows=1000 0.641822676 / 0.637994781 / +0.6%
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=1 rows=1000 0.544704224 / 0.533848905 / +2.0%
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=50 rows=1000 0.53032683 / 0.539979908 / -1.8%
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=1 rows=1000 0.541636437 / 0.552586204 / -2.0%
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=50 rows=1000 0.55051006 / 0.553478274 / -0.5%
vortex:parquet-zstd size/Arade 0.551185597 / 0.551185597 / -0.0%
vortex:parquet-zstd size/Bimbo 1.19261532 / 1.19261532 / +0.0%
vortex:parquet-zstd size/CMSprovider 1.11649863 / 1.11649863 / -0.0%
vortex:parquet-zstd size/Euro2016 1.33461969 / 1.33458157 / +0.0%
vortex:parquet-zstd size/Food 1.09107276 / 1.09107276 / -0.0%
vortex:parquet-zstd size/HashTags 1.46946859 / 1.46946859 / -0.0%
vortex:parquet-zstd size/TPC-H l_comment canonical 1.12837977 / 1.12837977 / -0.0%
vortex:parquet-zstd size/TPC-H l_comment chunked 1.12837977 / 1.12837977 / -0.0%
vortex:parquet-zstd size/taxi 0.904392267 / 0.904392267 / -0.0%
vortex:parquet-zstd size/wide table cols=100 chunks=1 rows=1000 1.0017975 / 1.0017975 / -0.0%
vortex:parquet-zstd size/wide table cols=100 chunks=50 rows=1000 1.0017975 / 1.0017975 / -0.0%
vortex:parquet-zstd size/wide table cols=1000 chunks=1 rows=1000 1.00017975 / 1.00017975 / -0.0%
vortex:parquet-zstd size/wide table cols=1000 chunks=50 rows=1000 1.00017975 / 1.00017975 / -0.0%
vortex:parquet-zstd size/wide table cols=10000 chunks=1 rows=1000 1.00040408 / 1.00040408 / -0.0%
vortex:parquet-zstd size/wide table cols=10000 chunks=50 rows=1000 1.00040408 / 1.00040408 / -0.0%
vortex / parquet / ns (1.004x ➖, 0↑ 0↓)
name ns (PR / base / %diff)
parquet_rs-zstd compress time/Arade 2249255873 / 2249342423 / -0.0%
parquet_rs-zstd compress time/Bimbo 11792381635 / 11856261171 / -0.5%
parquet_rs-zstd compress time/CMSprovider 6383552245 / 6384714530 / -0.0%
parquet_rs-zstd compress time/Euro2016 1077943271 / 1081915796 / -0.4%
parquet_rs-zstd compress time/Food 738866331 / 740418419 / -0.2%
parquet_rs-zstd compress time/HashTags 1827678602 / 1825644635 / +0.1%
parquet_rs-zstd compress time/TPC-H l_comment canonical 2618410305 / 2628381341 / -0.4%
parquet_rs-zstd compress time/TPC-H l_comment chunked 2628292592 / 2633220887 / -0.2%
parquet_rs-zstd compress time/taxi 1089857834 / 1089106665 / +0.1%
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 3976459 / 3954242 / +0.6%
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 3942045 / 4047945 / -2.6%
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 44572526 / 44484597 / +0.2%
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 45261458 / 44283776 / +2.2%
parquet_rs-zstd compress time/wide table cols=10000 chunks=1 rows=1000 466099856 / 458689596 / +1.6%
parquet_rs-zstd compress time/wide table cols=10000 chunks=50 rows=1000 461714563 / 452375388 / +2.1%
parquet_rs-zstd decompress time/Arade 556106205 / 556913129 / -0.1%
parquet_rs-zstd decompress time/Bimbo 1602465879 / 1602772898 / -0.0%
parquet_rs-zstd decompress time/CMSprovider 1360900218 / 1363723072 / -0.2%
parquet_rs-zstd decompress time/Euro2016 332312088 / 332428711 / -0.0%
parquet_rs-zstd decompress time/Food 169268489 / 169960467 / -0.4%
parquet_rs-zstd decompress time/HashTags 514829061 / 516632725 / -0.3%
parquet_rs-zstd decompress time/TPC-H l_comment canonical 497749323 / 497096436 / +0.1%
parquet_rs-zstd decompress time/TPC-H l_comment chunked 498410505 / 496852597 / +0.3%
parquet_rs-zstd decompress time/taxi 217525026 / 217564814 / -0.0%
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 2034019 / 2057071 / -1.1%
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 2049295 / 2113964 / -3.1%
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 23544218 / 23340445 / +0.9%
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 24183619 / 22982044 / +5.2%
parquet_rs-zstd decompress time/wide table cols=10000 chunks=1 rows=1000 252287185 / 241897599 / +4.3%
parquet_rs-zstd decompress time/wide table cols=10000 chunks=50 rows=1000 249505802 / 241196956 / +3.4%
vortex / parquet / bytes (1.000x ➖, 0↑ 0↓)
name bytes (PR / base / %diff)
parquet size/Arade 258014282 / 258014282 / +0.0%
parquet size/Bimbo 384517292 / 384517292 / +0.0%
parquet size/CMSprovider 376887911 / 376887911 / +0.0%
parquet size/Euro2016 122979049 / 122979049 / +0.0%
parquet size/Food 35699500 / 35699500 / +0.0%
parquet size/HashTags 133497590 / 133497590 / +0.0%
parquet size/TPC-H l_comment canonical 158358238 / 158358238 / +0.0%
parquet size/TPC-H l_comment chunked 158358238 / 158358238 / +0.0%
parquet size/taxi 55283635 / 55283635 / +0.0%
parquet size/wide table cols=100 chunks=1 rows=1000 932404 / 932404 / +0.0%
parquet size/wide table cols=100 chunks=50 rows=1000 932404 / 932404 / +0.0%
parquet size/wide table cols=1000 chunks=1 rows=1000 9324004 / 9324004 / +0.0%
parquet size/wide table cols=1000 chunks=50 rows=1000 9324004 / 9324004 / +0.0%
parquet size/wide table cols=10000 chunks=1 rows=1000 93240004 / 93240004 / +0.0%
parquet size/wide table cols=10000 chunks=50 rows=1000 93240004 / 93240004 / +0.0%
vortex / arrow-ipc / ns (1.019x ➖, 0↑ 5↓)
name ns (PR / base / %diff)
arrow-ipc compress time/Arade 99026405 / 100526703 / -1.5%
arrow-ipc compress time/Bimbo 617561377 / 632004766 / -2.3%
arrow-ipc compress time/CMSprovider 382618785 / 388818942 / -1.6%
arrow-ipc compress time/Euro2016 38487077 / 38778483 / -0.8%
arrow-ipc compress time/Food 33677410 / 34273067 / -1.7%
arrow-ipc compress time/HashTags 78720810 / 79055014 / -0.4%
arrow-ipc compress time/TPC-H l_comment canonical 950986525 / 952545520 / -0.2%
arrow-ipc compress time/TPC-H l_comment chunked 953313401 / 958239676 / -0.5%
arrow-ipc compress time/taxi 74614639 / 75598596 / -1.3%
arrow-ipc compress time/wide table cols=100 chunks=1 rows=1000 201237 / 213106 / -5.6%
arrow-ipc compress time/wide table cols=100 chunks=50 rows=1000 203105 / 203702 / -0.3%
arrow-ipc compress time/wide table cols=1000 chunks=1 rows=1000 2247220 / 2273386 / -1.2%
arrow-ipc compress time/wide table cols=1000 chunks=50 rows=1000 2228630 / 2276210 / -2.1%
arrow-ipc compress time/wide table cols=10000 chunks=1 rows=1000 25668976 / 25869661 / -0.8%
arrow-ipc compress time/wide table cols=10000 chunks=50 rows=1000 25624803 / 25846631 / -0.9%
arrow-ipc decompress time/Arade 182237870 / 142125027 / +28.2% 🔴
arrow-ipc decompress time/Bimbo 410652693 / 432178101 / -5.0%
arrow-ipc decompress time/CMSprovider 583097068 / 476421809 / +22.4% 🔴
arrow-ipc decompress time/Euro2016 74646465 / 64155739 / +16.4% 🔴
arrow-ipc decompress time/Food 58223308 / 49190058 / +18.4% 🔴
arrow-ipc decompress time/HashTags 142765023 / 122222178 / +16.8% 🔴
arrow-ipc decompress time/TPC-H l_comment canonical 635637203 / 641278112 / -0.9%
arrow-ipc decompress time/TPC-H l_comment chunked 626036699 / 641921911 / -2.5%
arrow-ipc decompress time/taxi 48072234 / 45692253 / +5.2%
arrow-ipc decompress time/wide table cols=100 chunks=1 rows=1000 322501 / 320811 / +0.5%
arrow-ipc decompress time/wide table cols=100 chunks=50 rows=1000 316960 / 320196 / -1.0%
arrow-ipc decompress time/wide table cols=1000 chunks=1 rows=1000 3485747 / 3558015 / -2.0%
arrow-ipc decompress time/wide table cols=1000 chunks=50 rows=1000 3501836 / 3676421 / -4.7%
arrow-ipc decompress time/wide table cols=10000 chunks=1 rows=1000 36359667 / 36985971 / -1.7%
arrow-ipc decompress time/wide table cols=10000 chunks=50 rows=1000 36326484 / 36763188 / -1.2%
vortex / arrow-ipc / bytes (1.000x ➖, 0↑ 0↓)
name bytes (PR / base / %diff)
arrow-ipc size/Arade 691765162 / 691765162 / +0.0%
arrow-ipc size/Bimbo 3279590114 / 3279590114 / +0.0%
arrow-ipc size/CMSprovider 2422259682 / 2422259682 / +0.0%
arrow-ipc size/Euro2016 360582666 / 360582666 / +0.0%
arrow-ipc size/Food 255782338 / 255782338 / +0.0%
arrow-ipc size/HashTags 704785810 / 704785810 / +0.0%
arrow-ipc size/TPC-H l_comment canonical 7317023586 / 7317023586 / +0.0%
arrow-ipc size/TPC-H l_comment chunked 7317023586 / 7317023586 / +0.0%
arrow-ipc size/taxi 482329690 / 482329690 / +0.0%
arrow-ipc size/wide table cols=100 chunks=1 rows=1000 1257290 / 1257290 / +0.0%
arrow-ipc size/wide table cols=100 chunks=50 rows=1000 1257290 / 1257290 / +0.0%
arrow-ipc size/wide table cols=1000 chunks=1 rows=1000 12568506 / 12568506 / +0.0%
arrow-ipc size/wide table cols=1000 chunks=50 rows=1000 12568506 / 12568506 / +0.0%
arrow-ipc size/wide table cols=10000 chunks=1 rows=1000 125752506 / 125752506 / +0.0%
arrow-ipc size/wide table cols=10000 chunks=50 rows=1000 125752506 / 125752506 / +0.0%

@robert3005
robert3005 deleted the claude/planner-decode-slice-migration-xppbf1 branch November 10, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants