Skip to content

Run the FineWeb benchmark on the V2 scan - #11055

Merged
joseph-isaacs merged 1 commit into
developfrom
ji/elegant-bardeen-y7z1cu
Oct 30, 2026
Merged

joseph-isaacs merged 1 commit into
developfrom
ji/elegant-bardeen-y7z1cu

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Moves the FineWeb benchmark onto the planner-driven V2 scan for both DataFusion and DuckDB.

The bench orchestrator (vx-bench) now chooses the scan version per benchmark (#11049), so the change is one entry in BENCHMARK_SCAN_V2. The bench binaries no longer pick a scan per benchmark.

Changes

  • bench-orchestrator/bench_orchestrator/config.py: Benchmark.FINEWEB: True.

An earlier version of this PR also changed how DuckDB's V2 driver cuts a file into parts. Its 65,536-row blocks gave the one-file, ~1M-row FineWeb sample only 16 parts, which made DuckDB 1.3–2.5x slower than V1. #11047 has since made DuckDB V2 parts follow the file's layout splits, the same units a V1 scan hands its threads. That replaces the code those commits patched, so they are dropped.

Results

FineWeb NVMe, median ms. V1 is from develop run 41056498175; V2 is from run 41101644140, whose log reports Vortex scan: V2.

Q DuckDB V1 DuckDB V2 DataFusion V1 DataFusion V2
0 2.0 2.2 7.0 6.0
1 9.7 7.6 10.4 9.3
2 11.1 9.3 9.6 11.5
3 19.2 19.2 19.9 17.8
4 67.0 66.2 67.8 65.8
5 66.2 66.0 67.8 62.4
6 19.4 19.2 19.2 18.7
7 18.9 18.2 17.5 18.8
8 5.7 4.7 9.5 12.2
Total 219.2 212.5 228.7 222.5

DuckDB is at least as fast on every query except Q0, with Q1 and Q2 about 20% faster. DataFusion is faster in total but slower on Q2, Q7 and Q8: Q8 is 9.5 → 12.2 ms and Q2 is 9.6 → 11.5 ms. Each side is a single 5-iteration run.

Checks

  • uv run --with pytest pytest tests/test_executor.py in bench-orchestrator: 33 passed. This includes the test that every benchmark has a scan version.
  • uvx ruff check bench-orchestrator/bench_orchestrator/config.py: passes.
  • The SQL benchmark workflow was started by hand on this branch (pr preset). Its jobs are red only because the PR-comment steps have no PR context on a manually started run.

To reproduce locally:

uv run --project bench-orchestrator vx-bench prepare-data fineweb --format parquet,vortex
uv run --project bench-orchestrator vx-bench run fineweb --engine datafusion,duckdb --format parquet,vortex

🤖 Generated with Claude Code

https://claude.ai/code/session_01UeniBARcDfvVv4UVqkMzmH

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBARcDfvVv4UVqkMzmH
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs
joseph-isaacs force-pushed the ji/elegant-bardeen-y7z1cu branch from f489a37 to 929c02f Compare October 30, 2026 17:48
@joseph-isaacs joseph-isaacs changed the title Default FineWeb to V2 scans and split small wide-row files finer in DuckDB Run the FineWeb benchmark on the V2 scan Oct 30, 2026
@codspeed

codspeed Bot commented Oct 30, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 46.46%

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚡ 4 improved benchmarks
✅ 2314 untouched benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Walltime filter_neon[64] 3.1 µs 1.8 µs +69.16%
⚡ Walltime threshold_neon[64] 4 µs 2.6 µs +54.83%
⚡ Walltime threshold_neon[80] 4.7 µs 3.4 µs +38.36%
⚡ Walltime filter_neon[80] 3.4 µs 2.7 µs +26.97%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ji/elegant-bardeen-y7z1cu (929c02f) with develop (19dcd2f)

Open in CodSpeed

@joseph-isaacs joseph-isaacs added the action/bench-sql Run SQL benchmarks without Vortex Compact on this PR label Oct 30, 2026
@github-actions github-actions Bot removed the action/bench-sql Run SQL benchmarks without Vortex Compact on this PR label Oct 30, 2026
@joseph-isaacs joseph-isaacs added the changelog/chore A trivial change label Oct 30, 2026
@joseph-isaacs
joseph-isaacs enabled auto-merge (squash) October 30, 2026 18:37
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 929c02f clickbench-nvme 1 Explore Profiling Data
🟢 Done 929c02f polarsignals 1 Explore Profiling Data
🟢 Done 929c02f tpch-nvme-10 1 Explore Profiling Data
🟢 Done 929c02f fineweb 1 Explore Profiling Data
🟢 Done 929c02f tpcds-nvme 1 Explore Profiling Data
🟢 Done 929c02f tpch-s3 1 Explore Profiling Data
🟢 Done 929c02f tpch-nvme 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: PolarSignals Profiling 📖

Commits: PR 929c02f6 vs base 19dcd2f2
Vortex (geomean): hot 0.976x ➖ · cold 1.022x ➖


datafusion / vortex-file-compressed / ns (0.976x ➖, 1↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
polarsignals_q00/datafusion:vortex-file-compressed 44139276 / 44198107 / -0.1% 90068961 / 89439370 / +0.7% 0.49 / 0.49 / -0.8%
polarsignals_q01/datafusion:vortex-file-compressed 53042324 / 53423409 / -0.7% 122839225 / 135084640 / -9.1% 0.43 / 0.40 / +9.2%
polarsignals_q02/datafusion:vortex-file-compressed 12027484 / 11887196 / +1.2% 52983679 / 50980997 / +3.9% 0.23 / 0.23 / -2.6%
polarsignals_q03/datafusion:vortex-file-compressed 53665576 / 71391736 / -24.8% 🟢 118266684 / 117694461 / +0.5% 0.45 / 0.61 / -25.2%
polarsignals_q04/datafusion:vortex-file-compressed 9386556 / 9044253 / +3.8% 44247042 / 44519732 / -0.6% 0.21 / 0.20 / +4.4%
polarsignals_q05/datafusion:vortex-file-compressed 11531876 / 11570807 / -0.3% 49105598 / 46066653 / +6.6% 0.23 / 0.25 / -6.5%
polarsignals_q06/datafusion:vortex-file-compressed 23264586 / 23543970 / -1.2% 59631870 / 55935271 / +6.6% 0.39 / 0.42 / -7.3%
polarsignals_q07/datafusion:vortex-file-compressed 11030747 / 10790669 / +2.2% 47004218 / 43925484 / +7.0% 0.23 / 0.25 / -4.5%
polarsignals_q08/datafusion:vortex-file-compressed 102852352 / 104038217 / -1.1% 158166201 / 159248816 / -0.7% 0.65 / 0.65 / -0.5%
polarsignals_q09/datafusion:vortex-file-compressed 9965315 / 9853164 / +1.1% 44157763 / 40771950 / +8.3% 0.23 / 0.24 / -6.6%

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: FineWeb NVMe 📖

Commits: PR 929c02f6 vs base 19dcd2f2
Verdict: No clear signal (low confidence)
Attributed Vortex impact: -4.4%
Engines: DataFusion No clear signal (-3.1%, low confidence) · DuckDB No clear signal (-5.6%, low confidence)
Vortex (geomean): hot 0.956x ➖ · cold 0.910x ➖
Parquet (geomean): hot 1.000x ➖ · cold 0.940x ➖
Shifts: Parquet (control) -0.0% · Median polish -1.4%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.

datafusion / vortex-file-compressed / ns (0.967x ➖, 2↑ 2↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/datafusion:vortex-file-compressed 5990393 / 6914164 / -13.4% 🟢 18581724 / 15894016 / +16.9% 🔴 0.32 / 0.44 / -25.9%
fineweb_q01/datafusion:vortex-file-compressed 9988651 / 8207537 / +21.7% 🔴 39191758 / 42273090 / -7.3% 0.25 / 0.19 / +31.3%
fineweb_q02/datafusion:vortex-file-compressed 10999980 / 9825386 / +12.0% 🔴 39619081 / 45991505 / -13.9% 🟢 0.28 / 0.21 / +30.0%
fineweb_q03/datafusion:vortex-file-compressed 17992948 / 19985364 / -10.0% 101387206 / 112849457 / -10.2% 🟢 0.18 / 0.18 / +0.2%
fineweb_q04/datafusion:vortex-file-compressed 65639895 / 67072209 / -2.1% 143499310 / 154793659 / -7.3% 0.46 / 0.43 / +5.6%
fineweb_q05/datafusion:vortex-file-compressed 63323873 / 67066206 / -5.6% 139659305 / 152013803 / -8.1% 0.45 / 0.44 / +2.8%
fineweb_q06/datafusion:vortex-file-compressed 18164376 / 19077528 / -4.8% 101073430 / 116530631 / -13.3% 🟢 0.18 / 0.16 / +9.8%
fineweb_q07/datafusion:vortex-file-compressed 18249720 / 19804552 / -7.9% 102492002 / 111729640 / -8.3% 0.18 / 0.18 / +0.5%
fineweb_q08/datafusion:vortex-file-compressed 7911984 / 9231922 / -14.3% 🟢 30961655 / 44729078 / -30.8% 🟢 0.26 / 0.21 / +23.8%
datafusion / parquet / ns (0.998x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/datafusion:parquet 9514717 / 9298051 / +2.3% 32460625 / 32161720 / +0.9% 0.29 / 0.29 / +1.4%
fineweb_q01/datafusion:parquet 74620526 / 75948676 / -1.7% 145898216 / 158050833 / -7.7% 0.51 / 0.48 / +6.4%
fineweb_q02/datafusion:parquet 75472046 / 74828561 / +0.9% 148740976 / 169180315 / -12.1% 🟢 0.51 / 0.44 / +14.7%
fineweb_q03/datafusion:parquet 73281701 / 75375235 / -2.8% 145962367 / 164650672 / -11.4% 🟢 0.50 / 0.46 / +9.7%
fineweb_q04/datafusion:parquet 82471712 / 82536988 / -0.1% 146294904 / 162124376 / -9.8% 0.56 / 0.51 / +10.7%
fineweb_q05/datafusion:parquet 80216022 / 80616398 / -0.5% 149985873 / 162946673 / -8.0% 0.53 / 0.49 / +8.1%
fineweb_q06/datafusion:parquet 75085753 / 74993748 / +0.1% 148092610 / 167327373 / -11.5% 🟢 0.51 / 0.45 / +13.1%
fineweb_q07/datafusion:parquet 74353264 / 74703466 / -0.5% 140995870 / 161875074 / -12.9% 🟢 0.53 / 0.46 / +14.3%
fineweb_q08/datafusion:parquet 73723408 / 73652338 / +0.1% 143719716 / 166993254 / -13.9% 🟢 0.51 / 0.44 / +16.3%
duckdb / vortex-file-compressed / ns (0.946x ➖, 3↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/duckdb:vortex-file-compressed 2086428 / 1992354 / +4.7% 7365774 / 7167288 / +2.8% 0.28 / 0.28 / +1.9%
fineweb_q01/duckdb:vortex-file-compressed 7859138 / 9302676 / -15.5% 🟢 25165002 / 33383961 / -24.6% 🟢 0.31 / 0.28 / +12.1%
fineweb_q02/duckdb:vortex-file-compressed 9472856 / 11086372 / -14.6% 🟢 27674477 / 32959285 / -16.0% 🟢 0.34 / 0.34 / +1.8%
fineweb_q03/duckdb:vortex-file-compressed 19321640 / 20108287 / -3.9% 87305346 / 93189312 / -6.3% 0.22 / 0.22 / +2.6%
fineweb_q04/duckdb:vortex-file-compressed 65590182 / 67281247 / -2.5% 103258583 / 109561330 / -5.8% 0.64 / 0.61 / +3.4%
fineweb_q05/duckdb:vortex-file-compressed 65090650 / 66431544 / -2.0% 101616748 / 106341165 / -4.4% 0.64 / 0.62 / +2.5%
fineweb_q06/duckdb:vortex-file-compressed 19084682 / 19036190 / +0.3% 87246622 / 93820093 / -7.0% 0.22 / 0.20 / +7.8%
fineweb_q07/duckdb:vortex-file-compressed 18355122 / 18786800 / -2.3% 87035600 / 90565938 / -3.9% 0.21 / 0.21 / +1.7%
fineweb_q08/duckdb:vortex-file-compressed 4806902 / 5376968 / -10.6% 🟢 12501879 / 13261327 / -5.7% 0.38 / 0.41 / -5.2%
duckdb / parquet / ns (1.002x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/duckdb:parquet 23835088 / 23973552 / -0.6% 28429006 / 28107016 / +1.1% 0.84 / 0.85 / -1.7%
fineweb_q01/duckdb:parquet 55854909 / 56632112 / -1.4% 73308705 / 75490626 / -2.9% 0.76 / 0.75 / +1.6%
fineweb_q02/duckdb:parquet 59871924 / 61085010 / -2.0% 76976545 / 84868736 / -9.3% 0.78 / 0.72 / +8.1%
fineweb_q03/duckdb:parquet 121287214 / 119366500 / +1.6% 182557118 / 185364019 / -1.5% 0.66 / 0.64 / +3.2%
fineweb_q04/duckdb:parquet 162699334 / 161989686 / +0.4% 209238221 / 214789038 / -2.6% 0.78 / 0.75 / +3.1%
fineweb_q05/duckdb:parquet 161541017 / 160108938 / +0.9% 202073466 / 209160474 / -3.4% 0.80 / 0.77 / +4.4%
fineweb_q06/duckdb:parquet 90615862 / 87576319 / +3.5% 152041023 / 148207180 / +2.6% 0.60 / 0.59 / +0.9%
fineweb_q07/duckdb:parquet 93004205 / 92192530 / +0.9% 143483883 / 148218580 / -3.2% 0.65 / 0.62 / +4.2%
fineweb_q08/duckdb:parquet 29861540 / 30171232 / -1.0% 31540346 / 31697555 / -0.5% 0.95 / 0.95 / -0.5%

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on NVME 📖

Commits: PR 929c02f6 vs base 19dcd2f2
Verdict: No clear signal (low confidence)
Attributed Vortex impact: -0.6%
Engines: DataFusion No clear signal (-1.0%, low confidence) · DuckDB No clear signal (-0.2%, low confidence)
Vortex (geomean): hot 0.995x ➖ · cold 1.017x ➖
Parquet (geomean): hot 1.001x ➖ · cold 1.012x ➖
Shifts: Parquet (control) +0.1% · Median polish -0.1%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.

datafusion / vortex-file-compressed / ns (0.993x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-file-compressed 15319504 / 15244796 / +0.5% 44989930 / 43107623 / +4.4% 0.34 / 0.35 / -3.7%
tpch_q02/datafusion:vortex-file-compressed 16553540 / 16785369 / -1.4% 35442255 / 35606088 / -0.5% 0.47 / 0.47 / -0.9%
tpch_q03/datafusion:vortex-file-compressed 19601763 / 19704057 / -0.5% 49704534 / 46014863 / +8.0% 0.39 / 0.43 / -7.9%
tpch_q04/datafusion:vortex-file-compressed 10369995 / 10379829 / -0.1% 34384221 / 32939078 / +4.4% 0.30 / 0.32 / -4.3%
tpch_q05/datafusion:vortex-file-compressed 36738596 / 37109828 / -1.0% 65808959 / 66535663 / -1.1% 0.56 / 0.56 / +0.1%
tpch_q06/datafusion:vortex-file-compressed 6075810 / 5996442 / +1.3% 29058664 / 25090471 / +15.8% 🔴 0.21 / 0.24 / -12.5%
tpch_q07/datafusion:vortex-file-compressed 51557385 / 51590471 / -0.1% 78717234 / 75221258 / +4.6% 0.65 / 0.69 / -4.5%
tpch_q08/datafusion:vortex-file-compressed 51841421 / 51934499 / -0.2% 77683546 / 79965898 / -2.9% 0.67 / 0.65 / +2.8%
tpch_q09/datafusion:vortex-file-compressed 47507226 / 47546136 / -0.1% 78540316 / 79942083 / -1.8% 0.60 / 0.59 / +1.7%
tpch_q10/datafusion:vortex-file-compressed 16977301 / 17645205 / -3.8% 51353645 / 47530092 / +8.0% 0.33 / 0.37 / -10.9%
tpch_q11/datafusion:vortex-file-compressed 14092303 / 14039900 / +0.4% 35186478 / 33161470 / +6.1% 0.40 / 0.42 / -5.4%
tpch_q12/datafusion:vortex-file-compressed 18231997 / 18302956 / -0.4% 42797949 / 43729009 / -2.1% 0.43 / 0.42 / +1.8%
tpch_q13/datafusion:vortex-file-compressed 18954521 / 19620384 / -3.4% 40176792 / 41663145 / -3.6% 0.47 / 0.47 / +0.2%
tpch_q14/datafusion:vortex-file-compressed 14476491 / 14553341 / -0.5% 39490461 / 37184545 / +6.2% 0.37 / 0.39 / -6.3%
tpch_q15/datafusion:vortex-file-compressed 18627710 / 18610835 / +0.1% 44181053 / 43225258 / +2.2% 0.42 / 0.43 / -2.1%
tpch_q16/datafusion:vortex-file-compressed 18463221 / 18329773 / +0.7% 42237601 / 40570847 / +4.1% 0.44 / 0.45 / -3.2%
tpch_q17/datafusion:vortex-file-compressed 27917679 / 27783290 / +0.5% 60900367 / 58070075 / +4.9% 0.46 / 0.48 / -4.2%
tpch_q18/datafusion:vortex-file-compressed 36430692 / 36366608 / +0.2% 58812847 / 60484474 / -2.8% 0.62 / 0.60 / +3.0%
tpch_q19/datafusion:vortex-file-compressed 12121975 / 12404911 / -2.3% 39176100 / 37955343 / +3.2% 0.31 / 0.33 / -5.3%
tpch_q20/datafusion:vortex-file-compressed 20010602 / 20323772 / -1.5% 45583642 / 45113470 / +1.0% 0.44 / 0.45 / -2.6%
tpch_q21/datafusion:vortex-file-compressed 39347308 / 40850950 / -3.7% 71561756 / 69381261 / +3.1% 0.55 / 0.59 / -6.6%
tpch_q22/datafusion:vortex-file-compressed 10709731 / 10664079 / +0.4% 31620401 / 31422514 / +0.6% 0.34 / 0.34 / -0.2%
datafusion / parquet / ns (1.003x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:parquet 72165896 / 72197453 / -0.0% 91514134 / 88021582 / +4.0% 0.79 / 0.82 / -3.9%
tpch_q02/datafusion:parquet 76773601 / 76113619 / +0.9% 95572819 / 91783594 / +4.1% 0.80 / 0.83 / -3.1%
tpch_q03/datafusion:parquet 84121210 / 83587244 / +0.6% 104718248 / 104079904 / +0.6% 0.80 / 0.80 / +0.0%
tpch_q04/datafusion:parquet 40769559 / 41074548 / -0.7% 59513718 / 58708424 / +1.4% 0.69 / 0.70 / -2.1%
tpch_q05/datafusion:parquet 112875855 / 111873269 / +0.9% 140085025 / 138232292 / +1.3% 0.81 / 0.81 / -0.4%
tpch_q06/datafusion:parquet 32460635 / 32164141 / +0.9% 50044835 / 48132854 / +4.0% 0.65 / 0.67 / -2.9%
tpch_q07/datafusion:parquet 114646730 / 115287140 / -0.6% 144408050 / 138852302 / +4.0% 0.79 / 0.83 / -4.4%
tpch_q08/datafusion:parquet 119802108 / 119975551 / -0.1% 146722795 / 146061083 / +0.5% 0.82 / 0.82 / -0.6%
tpch_q09/datafusion:parquet 140271919 / 140206080 / +0.0% 170927185 / 167370100 / +2.1% 0.82 / 0.84 / -2.0%
tpch_q10/datafusion:parquet 107304576 / 106687730 / +0.6% 131914690 / 129115049 / +2.2% 0.81 / 0.83 / -1.6%
tpch_q11/datafusion:parquet 60634606 / 60132019 / +0.8% 83590683 / 80765575 / +3.5% 0.73 / 0.74 / -2.6%
tpch_q12/datafusion:parquet 65097874 / 65627851 / -0.8% 90372742 / 87684783 / +3.1% 0.72 / 0.75 / -3.8%
tpch_q13/datafusion:parquet 147307549 / 147425213 / -0.1% 168027284 / 165702279 / +1.4% 0.88 / 0.89 / -1.5%
tpch_q14/datafusion:parquet 47635206 / 47798012 / -0.3% 73221263 / 69229230 / +5.8% 0.65 / 0.69 / -5.8%
tpch_q15/datafusion:parquet 74945242 / 74426434 / +0.7% 97808109 / 92888867 / +5.3% 0.77 / 0.80 / -4.4%
tpch_q16/datafusion:parquet 43997353 / 43982597 / +0.0% 60729355 / 60662108 / +0.1% 0.72 / 0.73 / -0.1%
tpch_q17/datafusion:parquet 124789753 / 123982033 / +0.7% 150122772 / 146222630 / +2.7% 0.83 / 0.85 / -2.0%
tpch_q18/datafusion:parquet 123709377 / 123784690 / -0.1% 147153181 / 145985967 / +0.8% 0.84 / 0.85 / -0.9%
tpch_q19/datafusion:parquet 58338496 / 58291246 / +0.1% 83672220 / 82910665 / +0.9% 0.70 / 0.70 / -0.8%
tpch_q20/datafusion:parquet 78742330 / 78072120 / +0.9% 105960678 / 104292926 / +1.6% 0.74 / 0.75 / -0.7%
tpch_q21/datafusion:parquet 117350003 / 116406825 / +0.8% 138882239 / 140577249 / -1.2% 0.84 / 0.83 / +2.0%
tpch_q22/datafusion:parquet 52161461 / 51648092 / +1.0% 68702914 / 70957245 / -3.2% 0.76 / 0.73 / +4.3%
duckdb / vortex-file-compressed / ns (0.997x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-file-compressed 9566508 / 9331715 / +2.5% 21272977 / 22301445 / -4.6% 0.45 / 0.42 / +7.5%
tpch_q02/duckdb:vortex-file-compressed 15653211 / 15484601 / +1.1% 25599284 / 25552226 / +0.2% 0.61 / 0.61 / +0.9%
tpch_q03/duckdb:vortex-file-compressed 18535614 / 19032979 / -2.6% 36187579 / 34316634 / +5.5% 0.51 / 0.55 / -7.6%
tpch_q04/duckdb:vortex-file-compressed 12525995 / 12472751 / +0.4% 26470927 / 26219641 / +1.0% 0.47 / 0.48 / -0.5%
tpch_q05/duckdb:vortex-file-compressed 20560802 / 20655679 / -0.5% 38075650 / 36659669 / +3.9% 0.54 / 0.56 / -4.2%
tpch_q06/duckdb:vortex-file-compressed 4892568 / 4857931 / +0.7% 14213998 / 14308025 / -0.7% 0.34 / 0.34 / +1.4%
tpch_q07/duckdb:vortex-file-compressed 24139145 / 24022009 / +0.5% 40969105 / 42111704 / -2.7% 0.59 / 0.57 / +3.3%
tpch_q08/duckdb:vortex-file-compressed 21095065 / 20985996 / +0.5% 39746923 / 37397733 / +6.3% 0.53 / 0.56 / -5.4%
tpch_q09/duckdb:vortex-file-compressed 21770986 / 23200613 / -6.2% 38855434 / 39713549 / -2.2% 0.56 / 0.58 / -4.1%
tpch_q10/duckdb:vortex-file-compressed 22475237 / 22845856 / -1.6% 39262958 / 37304677 / +5.2% 0.57 / 0.61 / -6.5%
tpch_q11/duckdb:vortex-file-compressed 8176141 / 8326778 / -1.8% 17849346 / 17727782 / +0.7% 0.46 / 0.47 / -2.5%
tpch_q12/duckdb:vortex-file-compressed 14627061 / 14696796 / -0.5% 26068061 / 26761598 / -2.6% 0.56 / 0.55 / +2.2%
tpch_q13/duckdb:vortex-file-compressed 19696700 / 19458653 / +1.2% 32478765 / 31940922 / +1.7% 0.61 / 0.61 / -0.5%
tpch_q14/duckdb:vortex-file-compressed 11619950 / 11617118 / +0.0% 23950664 / 24005548 / -0.2% 0.49 / 0.48 / +0.3%
tpch_q15/duckdb:vortex-file-compressed 8707869 / 8531969 / +2.1% 21015079 / 19674383 / +6.8% 0.41 / 0.43 / -4.4%
tpch_q16/duckdb:vortex-file-compressed 16710603 / 16840450 / -0.8% 23328487 / 23631828 / -1.3% 0.72 / 0.71 / +0.5%
tpch_q17/duckdb:vortex-file-compressed 12489615 / 12519740 / -0.2% 28191879 / 28338798 / -0.5% 0.44 / 0.44 / +0.3%
tpch_q18/duckdb:vortex-file-compressed 24582644 / 24609179 / -0.1% 34991628 / 33693817 / +3.9% 0.70 / 0.73 / -3.8%
tpch_q19/duckdb:vortex-file-compressed 12238948 / 12334808 / -0.8% 26471612 / 30730234 / -13.9% 🟢 0.46 / 0.40 / +15.2%
tpch_q20/duckdb:vortex-file-compressed 14278541 / 14293355 / -0.1% 33664510 / 30174689 / +11.6% 🔴 0.42 / 0.47 / -10.5%
tpch_q21/duckdb:vortex-file-compressed 37104782 / 37652395 / -1.5% 56559811 / 58206800 / -2.8% 0.66 / 0.65 / +1.4%
tpch_q22/duckdb:vortex-file-compressed 16979893 / 16837835 / +0.8% 27846278 / 26922218 / +3.4% 0.61 / 0.63 / -2.5%
duckdb / parquet / ns (0.999x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:parquet 48407353 / 48307040 / +0.2% 55252247 / 54616876 / +1.2% 0.88 / 0.88 / -0.9%
tpch_q02/duckdb:parquet 48784845 / 48896101 / -0.2% 56514697 / 56077374 / +0.8% 0.86 / 0.87 / -1.0%
tpch_q03/duckdb:parquet 60728161 / 60904244 / -0.3% 69705672 / 68371951 / +2.0% 0.87 / 0.89 / -2.2%
tpch_q04/duckdb:parquet 35397440 / 35226993 / +0.5% 44145031 / 44130480 / +0.0% 0.80 / 0.80 / +0.5%
tpch_q05/duckdb:parquet 67891962 / 68007656 / -0.2% 78950541 / 77731599 / +1.6% 0.86 / 0.87 / -1.7%
tpch_q06/duckdb:parquet 22147557 / 21889190 / +1.2% 27727802 / 27655555 / +0.3% 0.80 / 0.79 / +0.9%
tpch_q07/duckdb:parquet 64490194 / 64584662 / -0.1% 73670321 / 73435438 / +0.3% 0.88 / 0.88 / -0.5%
tpch_q08/duckdb:parquet 76450350 / 76297906 / +0.2% 88475561 / 87034681 / +1.7% 0.86 / 0.88 / -1.4%
tpch_q09/duckdb:parquet 108600265 / 108722543 / -0.1% 120642091 / 120496821 / +0.1% 0.90 / 0.90 / -0.2%
tpch_q10/duckdb:parquet 106717635 / 106897948 / -0.2% 119838559 / 116850249 / +2.6% 0.89 / 0.91 / -2.7%
tpch_q11/duckdb:parquet 29965575 / 29838442 / +0.4% 35725059 / 35529130 / +0.6% 0.84 / 0.84 / -0.1%
tpch_q12/duckdb:parquet 38302444 / 38296481 / +0.0% 44827084 / 45197180 / -0.8% 0.85 / 0.85 / +0.8%
tpch_q13/duckdb:parquet 175766754 / 175842570 / -0.0% 183268233 / 182383123 / +0.5% 0.96 / 0.96 / -0.5%
tpch_q14/duckdb:parquet 43146328 / 43217986 / -0.2% 52124603 / 52537443 / -0.8% 0.83 / 0.82 / +0.6%
tpch_q15/duckdb:parquet 33046717 / 33055371 / -0.0% 42147549 / 41534637 / +1.5% 0.78 / 0.80 / -1.5%
tpch_q16/duckdb:parquet 48350429 / 48446976 / -0.2% 53799034 / 53452908 / +0.6% 0.90 / 0.91 / -0.8%
tpch_q17/duckdb:parquet 48343306 / 48382020 / -0.1% 58118035 / 57851431 / +0.5% 0.83 / 0.84 / -0.5%
tpch_q18/duckdb:parquet 75270960 / 75180930 / +0.1% 82753822 / 83106544 / -0.4% 0.91 / 0.90 / +0.5%
tpch_q19/duckdb:parquet 49378889 / 50581915 / -2.4% 57243673 / 60744521 / -5.8% 0.86 / 0.83 / +3.6%
tpch_q20/duckdb:parquet 63953987 / 64115182 / -0.3% 75322084 / 73707959 / +2.2% 0.85 / 0.87 / -2.4%
tpch_q21/duckdb:parquet 107517736 / 108525558 / -0.9% 117999209 / 116925506 / +0.9% 0.91 / 0.93 / -1.8%
tpch_q22/duckdb:parquet 51285152 / 51338913 / -0.1% 60034357 / 59925342 / +0.2% 0.85 / 0.86 / -0.3%

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=10 on NVME 📖

Commits: PR 929c02f6 vs base 19dcd2f2
Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.0%
Engines: DataFusion No clear signal (-0.7%, low confidence) · DuckDB No clear signal (+0.8%, low confidence)
Vortex (geomean): hot 0.999x ➖ · cold 1.033x ➖
Parquet (geomean): hot 0.999x ➖ · cold 1.005x ➖
Shifts: Parquet (control) -0.1% · Median polish -0.1%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.

datafusion / vortex-file-compressed / ns (0.994x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-file-compressed 89982668 / 89255051 / +0.8% 151524804 / 151798817 / -0.2% 0.59 / 0.59 / +1.0%
tpch_q02/datafusion:vortex-file-compressed 63880000 / 65015788 / -1.7% 97073340 / 94942848 / +2.2% 0.66 / 0.68 / -3.9%
tpch_q03/datafusion:vortex-file-compressed 63471214 / 63658261 / -0.3% 117837548 / 111980694 / +5.2% 0.54 / 0.57 / -5.2%
tpch_q04/datafusion:vortex-file-compressed 33853930 / 34921471 / -3.1% 76053132 / 74731673 / +1.8% 0.45 / 0.47 / -4.7%
tpch_q05/datafusion:vortex-file-compressed 119329426 / 118970563 / +0.3% 172488598 / 186045364 / -7.3% 0.69 / 0.64 / +8.2%
tpch_q06/datafusion:vortex-file-compressed 10529026 / 10394960 / +1.3% 59781338 / 58729342 / +1.8% 0.18 / 0.18 / -0.5%
tpch_q07/datafusion:vortex-file-compressed 134134435 / 131980165 / +1.6% 196281520 / 199639612 / -1.7% 0.68 / 0.66 / +3.4%
tpch_q08/datafusion:vortex-file-compressed 142118188 / 142366362 / -0.2% 216730078 / 219277583 / -1.2% 0.66 / 0.65 / +1.0%
tpch_q09/datafusion:vortex-file-compressed 185672706 / 187171657 / -0.8% 271247875 / 257992349 / +5.1% 0.68 / 0.73 / -5.6%
tpch_q10/datafusion:vortex-file-compressed 70023553 / 70744077 / -1.0% 134825903 / 137653868 / -2.1% 0.52 / 0.51 / +1.1%
tpch_q11/datafusion:vortex-file-compressed 42921115 / 43769055 / -1.9% 68667538 / 70094330 / -2.0% 0.63 / 0.62 / +0.1%
tpch_q12/datafusion:vortex-file-compressed 35616105 / 35277279 / +1.0% 84764001 / 79113006 / +7.1% 0.42 / 0.45 / -5.8%
tpch_q13/datafusion:vortex-file-compressed 48907951 / 48858778 / +0.1% 84272946 / 83565410 / +0.8% 0.58 / 0.58 / -0.7%
tpch_q14/datafusion:vortex-file-compressed 21276376 / 21868695 / -2.7% 90327278 / 86598582 / +4.3% 0.24 / 0.25 / -6.7%
tpch_q15/datafusion:vortex-file-compressed 33334136 / 33975185 / -1.9% 91795467 / 92668034 / -0.9% 0.36 / 0.37 / -1.0%
tpch_q16/datafusion:vortex-file-compressed 37465655 / 37287766 / +0.5% 67644701 / 67877977 / -0.3% 0.55 / 0.55 / +0.8%
tpch_q17/datafusion:vortex-file-compressed 174438030 / 175172740 / -0.4% 229070812 / 220666595 / +3.8% 0.76 / 0.79 / -4.1%
tpch_q18/datafusion:vortex-file-compressed 202883090 / 206146884 / -1.6% 270203417 / 273649921 / -1.3% 0.75 / 0.75 / -0.3%
tpch_q19/datafusion:vortex-file-compressed 29793745 / 30422100 / -2.1% 91440725 / 92046105 / -0.7% 0.33 / 0.33 / -1.4%
tpch_q20/datafusion:vortex-file-compressed 60571301 / 61082053 / -0.8% 117878460 / 118547042 / -0.6% 0.51 / 0.52 / -0.3%
tpch_q21/datafusion:vortex-file-compressed 178309253 / 178607107 / -0.2% 236293608 / 241924423 / -2.3% 0.75 / 0.74 / +2.2%
tpch_q22/datafusion:vortex-file-compressed 24974865 / 24864567 / +0.4% 55673562 / 53580925 / +3.9% 0.45 / 0.46 / -3.3%
datafusion / parquet / ns (1.001x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:parquet 87840441 / 87368696 / +0.5% 127110167 / 127861995 / -0.6% 0.69 / 0.68 / +1.1%
tpch_q02/datafusion:parquet 177028220 / 174083826 / +1.7% 198762502 / 198946077 / -0.1% 0.89 / 0.88 / +1.8%
tpch_q03/datafusion:parquet 138331615 / 137844778 / +0.4% 184952615 / 183410886 / +0.8% 0.75 / 0.75 / -0.5%
tpch_q04/datafusion:parquet 59123269 / 58571544 / +0.9% 90936734 / 92920319 / -2.1% 0.65 / 0.63 / +3.1%
tpch_q05/datafusion:parquet 170033917 / 169683774 / +0.2% 227886368 / 224735109 / +1.4% 0.75 / 0.76 / -1.2%
tpch_q06/datafusion:parquet 37105824 / 37122976 / -0.0% 75829612 / 74947577 / +1.2% 0.49 / 0.50 / -1.2%
tpch_q07/datafusion:parquet 218270636 / 219518387 / -0.6% 275430272 / 275445618 / -0.0% 0.79 / 0.80 / -0.6%
tpch_q08/datafusion:parquet 221170887 / 222240942 / -0.5% 283951284 / 291393698 / -2.6% 0.78 / 0.76 / +2.1%
tpch_q09/datafusion:parquet 315586810 / 315267250 / +0.1% 380378968 / 378803120 / +0.4% 0.83 / 0.83 / -0.3%
tpch_q10/datafusion:parquet 321917333 / 321445702 / +0.1% 386582759 / 378830252 / +2.0% 0.83 / 0.85 / -1.9%
tpch_q11/datafusion:parquet 126319776 / 126573780 / -0.2% 149323140 / 151706161 / -1.6% 0.85 / 0.83 / +1.4%
tpch_q12/datafusion:parquet 84241973 / 84151856 / +0.1% 120776433 / 119202767 / +1.3% 0.70 / 0.71 / -1.2%
tpch_q13/datafusion:parquet 216667076 / 216495289 / +0.1% 246844403 / 244560647 / +0.9% 0.88 / 0.89 / -0.8%
tpch_q14/datafusion:parquet 84673202 / 88496640 / -4.3% 183356406 / 201728581 / -9.1% 0.46 / 0.44 / +5.3%
tpch_q15/datafusion:parquet 107236809 / 106076789 / +1.1% 153595494 / 155328519 / -1.1% 0.70 / 0.68 / +2.2%
tpch_q16/datafusion:parquet 98243250 / 95964632 / +2.4% 127105416 / 120010701 / +5.9% 0.77 / 0.80 / -3.3%
tpch_q17/datafusion:parquet 214134463 / 213016847 / +0.5% 260559428 / 258511536 / +0.8% 0.82 / 0.82 / -0.3%
tpch_q18/datafusion:parquet 271454020 / 271451611 / +0.0% 309235408 / 314651552 / -1.7% 0.88 / 0.86 / +1.8%
tpch_q19/datafusion:parquet 108916193 / 108017781 / +0.8% 164690762 / 169517847 / -2.8% 0.66 / 0.64 / +3.8%
tpch_q20/datafusion:parquet 189210739 / 190210191 / -0.5% 252713751 / 240188488 / +5.2% 0.75 / 0.79 / -5.5%
tpch_q21/datafusion:parquet 244840587 / 244828138 / +0.0% 318586546 / 308775289 / +3.2% 0.77 / 0.79 / -3.1%
tpch_q22/datafusion:parquet 154022799 / 153559494 / +0.3% 180943078 / 178194678 / +1.5% 0.85 / 0.86 / -1.2%
duckdb / vortex-file-compressed / ns (1.004x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-file-compressed 33068523 / 34628583 / -4.5% 73695427 / 63148371 / +16.7% 🔴 0.45 / 0.55 / -18.2%
tpch_q02/duckdb:vortex-file-compressed 28359867 / 28265284 / +0.3% 43461066 / 42869080 / +1.4% 0.65 / 0.66 / -1.0%
tpch_q03/duckdb:vortex-file-compressed 52725811 / 52134304 / +1.1% 105438592 / 97373188 / +8.3% 0.50 / 0.54 / -6.6%
tpch_q04/duckdb:vortex-file-compressed 51596322 / 50388682 / +2.4% 89189621 / 82805613 / +7.7% 0.58 / 0.61 / -4.9%
tpch_q05/duckdb:vortex-file-compressed 68177746 / 69552039 / -2.0% 111331091 / 105896848 / +5.1% 0.61 / 0.66 / -6.8%
tpch_q06/duckdb:vortex-file-compressed 15772146 / 15655028 / +0.7% 52991348 / 49233880 / +7.6% 0.30 / 0.32 / -6.4%
tpch_q07/duckdb:vortex-file-compressed 55942640 / 55924649 / +0.0% 112631903 / 102703022 / +9.7% 0.50 / 0.54 / -8.8%
tpch_q08/duckdb:vortex-file-compressed 87331543 / 87648586 / -0.4% 135710511 / 128820535 / +5.3% 0.64 / 0.68 / -5.4%
tpch_q09/duckdb:vortex-file-compressed 102334948 / 101895414 / +0.4% 160194971 / 153843362 / +4.1% 0.64 / 0.66 / -3.6%
tpch_q10/duckdb:vortex-file-compressed 94911048 / 95578746 / -0.7% 144900177 / 138864358 / +4.3% 0.66 / 0.69 / -4.8%
tpch_q11/duckdb:vortex-file-compressed 13422703 / 13359125 / +0.5% 29683631 / 29133955 / +1.9% 0.45 / 0.46 / -1.4%
tpch_q12/duckdb:vortex-file-compressed 45321798 / 44778474 / +1.2% 85567586 / 80033923 / +6.9% 0.53 / 0.56 / -5.3%
tpch_q13/duckdb:vortex-file-compressed 72663532 / 73185545 / -0.7% 101926494 / 95541761 / +6.7% 0.71 / 0.77 / -6.9%
tpch_q14/duckdb:vortex-file-compressed 28577535 / 28694401 / -0.4% 73590706 / 69184778 / +6.4% 0.39 / 0.41 / -6.4%
tpch_q15/duckdb:vortex-file-compressed 25132408 / 24917046 / +0.9% 66248141 / 61375732 / +7.9% 0.38 / 0.41 / -6.6%
tpch_q16/duckdb:vortex-file-compressed 49347391 / 47849508 / +3.1% 57835802 / 58544628 / -1.2% 0.85 / 0.82 / +4.4%
tpch_q17/duckdb:vortex-file-compressed 35268608 / 35386705 / -0.3% 68023458 / 65592778 / +3.7% 0.52 / 0.54 / -3.9%
tpch_q18/duckdb:vortex-file-compressed 111863638 / 105387928 / +6.1% 124373762 / 120554783 / +3.2% 0.90 / 0.87 / +2.9%
tpch_q19/duckdb:vortex-file-compressed 28133073 / 27879632 / +0.9% 76410329 / 72375475 / +5.6% 0.37 / 0.39 / -4.4%
tpch_q20/duckdb:vortex-file-compressed 41465175 / 41873332 / -1.0% 97634123 / 89530338 / +9.1% 0.42 / 0.47 / -9.2%
tpch_q21/duckdb:vortex-file-compressed 95470873 / 96352501 / -0.9% 150078428 / 143914571 / +4.3% 0.64 / 0.67 / -5.0%
tpch_q22/duckdb:vortex-file-compressed 33319520 / 32234491 / +3.4% 52682674 / 48211625 / +9.3% 0.63 / 0.67 / -5.4%
duckdb / parquet / ns (0.997x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:parquet 69086698 / 68884852 / +0.3% 88854686 / 90032929 / -1.3% 0.78 / 0.77 / +1.6%
tpch_q02/duckdb:parquet 87823202 / 87634295 / +0.2% 98793043 / 98800141 / -0.0% 0.89 / 0.89 / +0.2%
tpch_q03/duckdb:parquet 111025290 / 111393685 / -0.3% 142530728 / 143939319 / -1.0% 0.78 / 0.77 / +0.7%
tpch_q04/duckdb:parquet 77485589 / 78761710 / -1.6% 97836486 / 96907681 / +1.0% 0.79 / 0.81 / -2.6%
tpch_q05/duckdb:parquet 123162559 / 125473048 / -1.8% 157623086 / 157708809 / -0.1% 0.78 / 0.80 / -1.8%
tpch_q06/duckdb:parquet 36081772 / 36143926 / -0.2% 63947562 / 57665594 / +10.9% 🔴 0.56 / 0.63 / -10.0%
tpch_q07/duckdb:parquet 118884798 / 117157091 / +1.5% 153686986 / 155070011 / -0.9% 0.77 / 0.76 / +2.4%
tpch_q08/duckdb:parquet 148696712 / 149051786 / -0.2% 191998028 / 190406433 / +0.8% 0.77 / 0.78 / -1.1%
tpch_q09/duckdb:parquet 239052538 / 241388962 / -1.0% 276293198 / 284144788 / -2.8% 0.87 / 0.85 / +1.8%
tpch_q10/duckdb:parquet 316317251 / 316818482 / -0.2% 340754313 / 340358055 / +0.1% 0.93 / 0.93 / -0.3%
tpch_q11/duckdb:parquet 44124791 / 44058957 / +0.1% 51292622 / 51642228 / -0.7% 0.86 / 0.85 / +0.8%
tpch_q12/duckdb:parquet 71689659 / 72613282 / -1.3% 93355579 / 92369855 / +1.1% 0.77 / 0.79 / -2.3%
tpch_q13/duckdb:parquet 293294805 / 295602456 / -0.8% 310399104 / 311037836 / -0.2% 0.94 / 0.95 / -0.6%
tpch_q14/duckdb:parquet 103169417 / 102641437 / +0.5% 139812970 / 137537356 / +1.7% 0.74 / 0.75 / -1.1%
tpch_q15/duckdb:parquet 58673125 / 58562796 / +0.2% 87188393 / 87341612 / -0.2% 0.67 / 0.67 / +0.4%
tpch_q16/duckdb:parquet 101300632 / 101516941 / -0.2% 109378440 / 107924096 / +1.3% 0.93 / 0.94 / -1.5%
tpch_q17/duckdb:parquet 83356892 / 82982497 / +0.5% 119536383 / 116477291 / +2.6% 0.70 / 0.71 / -2.1%
tpch_q18/duckdb:parquet 159611780 / 162781413 / -1.9% 173029801 / 176107872 / -1.7% 0.92 / 0.92 / -0.2%
tpch_q19/duckdb:parquet 99206382 / 100644327 / -1.4% 134514839 / 129160526 / +4.1% 0.74 / 0.78 / -5.4%
tpch_q20/duckdb:parquet 148463774 / 147762874 / +0.5% 186118398 / 181776837 / +2.4% 0.80 / 0.81 / -1.9%
tpch_q21/duckdb:parquet 257355116 / 256842317 / +0.2% 279958896 / 275377695 / +1.7% 0.92 / 0.93 / -1.4%
tpch_q22/duckdb:parquet 169640186 / 169184411 / +0.3% 184562073 / 183870885 / +0.4% 0.92 / 0.92 / -0.1%

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-DS SF=1 on NVME 📖

Commits: PR 929c02f6 vs base 19dcd2f2
Verdict: No clear signal (low confidence)
Attributed Vortex impact: -0.5%
Engines: DataFusion No clear signal (-0.6%, low confidence) · DuckDB No clear signal (-0.5%, low confidence)
Vortex (geomean): hot 1.002x ➖ · cold 1.002x ➖
Parquet (geomean): hot 1.007x ➖ · cold 1.007x ➖
Shifts: Parquet (control) +0.7% · Median polish +0.2%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.

datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/datafusion:vortex-file-compressed 21058692 / 20662877 / +1.9% 38682213 / 38388298 / +0.8% 0.54 / 0.54 / +1.1%
tpcds_q02/datafusion:vortex-file-compressed 44688315 / 44071682 / +1.4% 66300947 / 64203849 / +3.3% 0.67 / 0.69 / -1.8%
tpcds_q03/datafusion:vortex-file-compressed 11086782 / 11095395 / -0.1% 28897656 / 26770394 / +7.9% 0.38 / 0.41 / -7.4%
tpcds_q04/datafusion:vortex-file-compressed 88237350 / 85937970 / +2.7% 120548992 / 120762373 / -0.2% 0.73 / 0.71 / +2.9%
tpcds_q05/datafusion:vortex-file-compressed 56271976 / 56220694 / +0.1% 77447000 / 76967535 / +0.6% 0.73 / 0.73 / -0.5%
tpcds_q06/datafusion:vortex-file-compressed 22573842 / 21976346 / +2.7% 42738829 / 43424391 / -1.6% 0.53 / 0.51 / +4.4%
tpcds_q07/datafusion:vortex-file-compressed 28612858 / 28753308 / -0.5% 51418046 / 51675554 / -0.5% 0.56 / 0.56 / +0.0%
tpcds_q08/datafusion:vortex-file-compressed 24683168 / 24541650 / +0.6% 43262384 / 43475451 / -0.5% 0.57 / 0.56 / +1.1%
tpcds_q09/datafusion:vortex-file-compressed 19918968 / 20105705 / -0.9% 36463487 / 37029178 / -1.5% 0.55 / 0.54 / +0.6%
tpcds_q10/datafusion:vortex-file-compressed 26478773 / 26771250 / -1.1% 49848049 / 50455373 / -1.2% 0.53 / 0.53 / +0.1%
tpcds_q11/datafusion:vortex-file-compressed 53834424 / 51942594 / +3.6% 82561638 / 83041366 / -0.6% 0.65 / 0.63 / +4.2%
tpcds_q12/datafusion:vortex-file-compressed 14756690 / 14909005 / -1.0% 32494515 / 32866199 / -1.1% 0.45 / 0.45 / +0.1%
tpcds_q13/datafusion:vortex-file-compressed 31048738 / 31125908 / -0.2% 58287071 / 56678644 / +2.8% 0.53 / 0.55 / -3.0%
tpcds_q14/datafusion:vortex-file-compressed 490270498 / 487173892 / +0.6% 546350664 / 549971662 / -0.7% 0.90 / 0.89 / +1.3%
tpcds_q15/datafusion:vortex-file-compressed 12300363 / 12454406 / -1.2% 32672364 / 33623513 / -2.8% 0.38 / 0.37 / +1.6%
tpcds_q16/datafusion:vortex-file-compressed 18158534 / 18526448 / -2.0% 38423854 / 38010837 / +1.1% 0.47 / 0.49 / -3.0%
tpcds_q17/datafusion:vortex-file-compressed 54985018 / 50769293 / +8.3% 76109982 / 75447269 / +0.9% 0.72 / 0.67 / +7.4%
tpcds_q18/datafusion:vortex-file-compressed 63668338 / 63588463 / +0.1% 86889607 / 85468521 / +1.7% 0.73 / 0.74 / -1.5%
tpcds_q19/datafusion:vortex-file-compressed 17017776 / 17047396 / -0.2% 39343360 / 38742283 / +1.6% 0.43 / 0.44 / -1.7%
tpcds_q20/datafusion:vortex-file-compressed 16325730 / 16464230 / -0.8% 37228193 / 35800079 / +4.0% 0.44 / 0.46 / -4.6%
tpcds_q21/datafusion:vortex-file-compressed 16103228 / 15940484 / +1.0% 38283461 / 37502958 / +2.1% 0.42 / 0.43 / -1.0%
tpcds_q22/datafusion:vortex-file-compressed 50483750 / 49830978 / +1.3% 71627828 / 73429530 / -2.5% 0.70 / 0.68 / +3.9%
tpcds_q23/datafusion:vortex-file-compressed 71592434 / 71127504 / +0.7% 95572773 / 95266102 / +0.3% 0.75 / 0.75 / +0.3%
tpcds_q24/datafusion:vortex-file-compressed 64949049 / 65827480 / -1.3% 87558352 / 87950060 / -0.4% 0.74 / 0.75 / -0.9%
tpcds_q25/datafusion:vortex-file-compressed 48103490 / 48552602 / -0.9% 75867134 / 69532625 / +9.1% 0.63 / 0.70 / -9.2%
tpcds_q26/datafusion:vortex-file-compressed 25546253 / 25883028 / -1.3% 47189527 / 45868290 / +2.9% 0.54 / 0.56 / -4.1%
tpcds_q27/datafusion:vortex-file-compressed 58683788 / 63281436 / -7.3% 81109821 / 80805852 / +0.4% 0.72 / 0.78 / -7.6%
tpcds_q28/datafusion:vortex-file-compressed 25659614 / 26229064 / -2.2% 40675694 / 40810592 / -0.3% 0.63 / 0.64 / -1.8%
tpcds_q29/datafusion:vortex-file-compressed 53888474 / 53880442 / +0.0% 78029522 / 75632558 / +3.2% 0.69 / 0.71 / -3.1%
tpcds_q30/datafusion:vortex-file-compressed 26186713 / 25882580 / +1.2% 44841899 / 45306891 / -1.0% 0.58 / 0.57 / +2.2%
tpcds_q31/datafusion:vortex-file-compressed 58473520 / 58167868 / +0.5% 83437259 / 83805563 / -0.4% 0.70 / 0.69 / +1.0%
tpcds_q32/datafusion:vortex-file-compressed 12313056 / 12486154 / -1.4% 29506940 / 29878584 / -1.2% 0.42 / 0.42 / -0.1%
tpcds_q33/datafusion:vortex-file-compressed 26972870 / 27470150 / -1.8% 51634391 / 50803280 / +1.6% 0.52 / 0.54 / -3.4%
tpcds_q34/datafusion:vortex-file-compressed 17689257 / 17355034 / +1.9% 36788375 / 37376510 / -1.6% 0.48 / 0.46 / +3.6%
tpcds_q35/datafusion:vortex-file-compressed 30206466 / 30165718 / +0.1% 55398102 / 54544146 / +1.6% 0.55 / 0.55 / -1.4%
tpcds_q36/datafusion:vortex-file-compressed 49240316 / 48307158 / +1.9% 72320923 / 67264121 / +7.5% 0.68 / 0.72 / -5.2%
tpcds_q37/datafusion:vortex-file-compressed 11696388 / 11857472 / -1.4% 35845500 / 34111657 / +5.1% 0.33 / 0.35 / -6.1%
tpcds_q38/datafusion:vortex-file-compressed 25491478 / 25514558 / -0.1% 48686195 / 48241747 / +0.9% 0.52 / 0.53 / -1.0%
tpcds_q39/datafusion:vortex-file-compressed 64017458 / 64890302 / -1.3% 89277811 / 89633580 / -0.4% 0.72 / 0.72 / -1.0%
tpcds_q40/datafusion:vortex-file-compressed 23440160 / 23494010 / -0.2% 43003248 / 43948484 / -2.2% 0.55 / 0.53 / +2.0%
tpcds_q41/datafusion:vortex-file-compressed 20257107 / 20495924 / -1.2% 32419210 / 32839414 / -1.3% 0.62 / 0.62 / +0.1%
tpcds_q42/datafusion:vortex-file-compressed 10985146 / 10944217 / +0.4% 26999693 / 26800473 / +0.7% 0.41 / 0.41 / -0.4%
tpcds_q43/datafusion:vortex-file-compressed 12493760 / 12235502 / +2.1% 32326394 / 32362966 / -0.1% 0.39 / 0.38 / +2.2%
tpcds_q44/datafusion:vortex-file-compressed 27852236 / 28950501 / -3.8% 51631772 / 53633308 / -3.7% 0.54 / 0.54 / -0.1%
tpcds_q45/datafusion:vortex-file-compressed 15811789 / 16086212 / -1.7% 34942506 / 36240197 / -3.6% 0.45 / 0.44 / +1.9%
tpcds_q46/datafusion:vortex-file-compressed 23034315 / 23330782 / -1.3% 48003754 / 46023788 / +4.3% 0.48 / 0.51 / -5.3%
tpcds_q47/datafusion:vortex-file-compressed 75736322 / 74638130 / +1.5% 107166721 / 104471851 / +2.6% 0.71 / 0.71 / -1.1%
tpcds_q48/datafusion:vortex-file-compressed 28544536 / 28355408 / +0.7% 53526757 / 52242427 / +2.5% 0.53 / 0.54 / -1.7%
tpcds_q49/datafusion:vortex-file-compressed 59047118 / 57975552 / +1.8% 79009038 / 80249136 / -1.5% 0.75 / 0.72 / +3.4%
tpcds_q50/datafusion:vortex-file-compressed 30525864 / 30773469 / -0.8% 54791694 / 53329113 / +2.7% 0.56 / 0.58 / -3.5%
tpcds_q51/datafusion:vortex-file-compressed 66136428 / 66370124 / -0.4% 88546879 / 87949849 / +0.7% 0.75 / 0.75 / -1.0%
tpcds_q52/datafusion:vortex-file-compressed 11316304 / 11303700 / +0.1% 27363509 / 28114971 / -2.7% 0.41 / 0.40 / +2.9%
tpcds_q53/datafusion:vortex-file-compressed 17703567 / 17817615 / -0.6% 37967687 / 39106120 / -2.9% 0.47 / 0.46 / +2.3%
tpcds_q54/datafusion:vortex-file-compressed 30980584 / 30815867 / +0.5% 55450695 / 53633024 / +3.4% 0.56 / 0.57 / -2.8%
tpcds_q55/datafusion:vortex-file-compressed 10769362 / 10947926 / -1.6% 26272963 / 26268641 / +0.0% 0.41 / 0.42 / -1.6%
tpcds_q56/datafusion:vortex-file-compressed 26068535 / 25893575 / +0.7% 48097320 / 48418478 / -0.7% 0.54 / 0.53 / +1.3%
tpcds_q57/datafusion:vortex-file-compressed 62815137 / 61658540 / +1.9% 89716995 / 88488887 / +1.4% 0.70 / 0.70 / +0.5%
tpcds_q58/datafusion:vortex-file-compressed 38150138 / 38139050 / +0.0% 62588785 / 62952715 / -0.6% 0.61 / 0.61 / +0.6%
tpcds_q59/datafusion:vortex-file-compressed 29643642 / 29773657 / -0.4% 55677465 / 54576681 / +2.0% 0.53 / 0.55 / -2.4%
tpcds_q60/datafusion:vortex-file-compressed 26139214 / 25763284 / +1.5% 46953627 / 47311516 / -0.8% 0.56 / 0.54 / +2.2%
tpcds_q61/datafusion:vortex-file-compressed 21207536 / 21212998 / -0.0% 45120675 / 43808218 / +3.0% 0.47 / 0.48 / -2.9%
tpcds_q62/datafusion:vortex-file-compressed 16493559 / 16226768 / +1.6% 35035303 / 34675741 / +1.0% 0.47 / 0.47 / +0.6%
tpcds_q63/datafusion:vortex-file-compressed 17505000 / 17479711 / +0.1% 37588390 / 38987535 / -3.6% 0.47 / 0.45 / +3.9%
tpcds_q64/datafusion:vortex-file-compressed 212993298 / 212632786 / +0.2% 247113428 / 245822306 / +0.5% 0.86 / 0.86 / -0.4%
tpcds_q65/datafusion:vortex-file-compressed 72717754 / 72326343 / +0.5% 93370332 / 92518043 / +0.9% 0.78 / 0.78 / -0.4%
tpcds_q66/datafusion:vortex-file-compressed 50791970 / 50841554 / -0.1% 74018035 / 72759471 / +1.7% 0.69 / 0.70 / -1.8%
tpcds_q67/datafusion:vortex-file-compressed 51294130 / 51321318 / -0.1% 82010393 / 84264967 / -2.7% 0.63 / 0.61 / +2.7%
tpcds_q68/datafusion:vortex-file-compressed 23019914 / 22754568 / +1.2% 49068140 / 46535563 / +5.4% 0.47 / 0.49 / -4.1%
tpcds_q69/datafusion:vortex-file-compressed 25481271 / 25748720 / -1.0% 48630798 / 47728418 / +1.9% 0.52 / 0.54 / -2.9%
tpcds_q70/datafusion:vortex-file-compressed 71203874 / 70768666 / +0.6% 94338052 / 94483406 / -0.2% 0.75 / 0.75 / +0.8%
tpcds_q71/datafusion:vortex-file-compressed 18265869 / 19265814 / -5.2% 40277704 / 40165351 / +0.3% 0.45 / 0.48 / -5.5%
tpcds_q72/datafusion:vortex-file-compressed 351563771 / 346150522 / +1.6% 381128724 / 397297658 / -4.1% 0.92 / 0.87 / +5.9%
tpcds_q73/datafusion:vortex-file-compressed 16053454 / 15934651 / +0.7% 35147679 / 35011866 / +0.4% 0.46 / 0.46 / +0.4%
tpcds_q74/datafusion:vortex-file-compressed 39893126 / 39780041 / +0.3% 68201091 / 66803203 / +2.1% 0.58 / 0.60 / -1.8%
tpcds_q75/datafusion:vortex-file-compressed 135582114 / 135714753 / -0.1% 160607209 / 162660370 / -1.3% 0.84 / 0.83 / +1.2%
tpcds_q76/datafusion:vortex-file-compressed 29344275 / 29291308 / +0.2% 51610356 / 47924547 / +7.7% 0.57 / 0.61 / -7.0%
tpcds_q77/datafusion:vortex-file-compressed 44177579 / 44660360 / -1.1% 67769736 / 65734489 / +3.1% 0.65 / 0.68 / -4.1%
tpcds_q78/datafusion:vortex-file-compressed 63876222 / 63632236 / +0.4% 92101407 / 94010583 / -2.0% 0.69 / 0.68 / +2.5%
tpcds_q79/datafusion:vortex-file-compressed 19981392 / 19924334 / +0.3% 41424403 / 42878743 / -3.4% 0.48 / 0.46 / +3.8%
tpcds_q80/datafusion:vortex-file-compressed 74195933 / 72991048 / +1.7% 99097971 / 100489903 / -1.4% 0.75 / 0.73 / +3.1%
tpcds_q81/datafusion:vortex-file-compressed 26573904 / 26190294 / +1.5% 45192920 / 44415165 / +1.8% 0.59 / 0.59 / -0.3%
tpcds_q82/datafusion:vortex-file-compressed 12038260 / 12373004 / -2.7% 33920211 / 33916690 / +0.0% 0.35 / 0.36 / -2.7%
tpcds_q83/datafusion:vortex-file-compressed 33242752 / 32776157 / +1.4% 52411091 / 50511779 / +3.8% 0.63 / 0.65 / -2.3%
tpcds_q84/datafusion:vortex-file-compressed 9966264 / 10014122 / -0.5% 26257777 / 26553087 / -1.1% 0.38 / 0.38 / +0.6%
tpcds_q85/datafusion:vortex-file-compressed 53180128 / 53044578 / +0.3% 79739734 / 80954298 / -1.5% 0.67 / 0.66 / +1.8%
tpcds_q86/datafusion:vortex-file-compressed 15653228 / 15658548 / -0.0% 34025889 / 32311558 / +5.3% 0.46 / 0.48 / -5.1%
tpcds_q87/datafusion:vortex-file-compressed 25941316 / 25551026 / +1.5% 48963340 / 46861588 / +4.5% 0.53 / 0.55 / -2.8%
tpcds_q88/datafusion:vortex-file-compressed 31645489 / 31550716 / +0.3% 52699642 / 54227953 / -2.8% 0.60 / 0.58 / +3.2%
tpcds_q89/datafusion:vortex-file-compressed 21280401 / 21217704 / +0.3% 42252283 / 40149772 / +5.2% 0.50 / 0.53 / -4.7%
tpcds_q90/datafusion:vortex-file-compressed 10179655 / 9912202 / +2.7% 25719340 / 24923081 / +3.2% 0.40 / 0.40 / -0.5%
tpcds_q91/datafusion:vortex-file-compressed 17077876 / 16331088 / +4.6% 36701279 / 36210118 / +1.4% 0.47 / 0.45 / +3.2%
tpcds_q92/datafusion:vortex-file-compressed 12935286 / 12712254 / +1.8% 28641747 / 29043054 / -1.4% 0.45 / 0.44 / +3.2%
tpcds_q93/datafusion:vortex-file-compressed 20612119 / 20005716 / +3.0% 42543639 / 40530909 / +5.0% 0.48 / 0.49 / -1.8%
tpcds_q94/datafusion:vortex-file-compressed 18509005 / 18455416 / +0.3% 36844863 / 37274216 / -1.2% 0.50 / 0.50 / +1.5%
tpcds_q95/datafusion:vortex-file-compressed 42423760 / 42223247 / +0.5% 62830735 / 61088035 / +2.9% 0.68 / 0.69 / -2.3%
tpcds_q96/datafusion:vortex-file-compressed 7383610 / 7643602 / -3.4% 24203019 / 24204839 / -0.0% 0.31 / 0.32 / -3.4%
tpcds_q97/datafusion:vortex-file-compressed 17002420 / 17142063 / -0.8% 37829660 / 36143924 / +4.7% 0.45 / 0.47 / -5.2%
tpcds_q98/datafusion:vortex-file-compressed 16386208 / 16025235 / +2.3% 37337370 / 36067472 / +3.5% 0.44 / 0.44 / -1.2%
tpcds_q99/datafusion:vortex-file-compressed 16218851 / 15816152 / +2.5% 34576417 / 35033983 / -1.3% 0.47 / 0.45 / +3.9%
datafusion / parquet / ns (1.008x ➖, 1↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/datafusion:parquet 43962344 / 41037398 / +7.1% 61086372 / 60468031 / +1.0% 0.72 / 0.68 / +6.0%
tpcds_q02/datafusion:parquet 55386749 / 55294162 / +0.2% 72763048 / 72584598 / +0.2% 0.76 / 0.76 / -0.1%
tpcds_q03/datafusion:parquet 19130574 / 19145511 / -0.1% 35168806 / 31737576 / +10.8% 🔴 0.54 / 0.60 / -9.8%
tpcds_q04/datafusion:parquet 180465549 / 179148961 / +0.7% 208302864 / 207193262 / +0.5% 0.87 / 0.86 / +0.2%
tpcds_q05/datafusion:parquet 69436068 / 67866314 / +2.3% 95153037 / 93269797 / +2.0% 0.73 / 0.73 / +0.3%
tpcds_q06/datafusion:parquet 34154986 / 34548628 / -1.1% 48791581 / 48650311 / +0.3% 0.70 / 0.71 / -1.4%
tpcds_q07/datafusion:parquet 47531924 / 47252786 / +0.6% 68848450 / 68512135 / +0.5% 0.69 / 0.69 / +0.1%
tpcds_q08/datafusion:parquet 31607378 / 31369000 / +0.8% 47528696 / 45820766 / +3.7% 0.67 / 0.68 / -2.9%
tpcds_q09/datafusion:parquet 30156500 / 34307494 / -12.1% 🟢 50509098 / 43105925 / +17.2% 🔴 0.60 / 0.80 / -25.0%
tpcds_q10/datafusion:parquet 44383630 / 44551387 / -0.4% 61774477 / 60617616 / +1.9% 0.72 / 0.73 / -2.2%
tpcds_q11/datafusion:parquet 100172706 / 98283752 / +1.9% 122058946 / 121127398 / +0.8% 0.82 / 0.81 / +1.1%
tpcds_q12/datafusion:parquet 23980335 / 24266474 / -1.2% 38316962 / 36327580 / +5.5% 0.63 / 0.67 / -6.3%
tpcds_q13/datafusion:parquet 46867438 / 46580310 / +0.6% 67727994 / 67994465 / -0.4% 0.69 / 0.69 / +1.0%
tpcds_q14/datafusion:parquet 535543944 / 535803457 / -0.0% 587902939 / 576399589 / +2.0% 0.91 / 0.93 / -2.0%
tpcds_q15/datafusion:parquet 23392193 / 23987052 / -2.5% 39561281 / 38772857 / +2.0% 0.59 / 0.62 / -4.4%
tpcds_q16/datafusion:parquet 37101018 / 37385275 / -0.8% 53307705 / 50808144 / +4.9% 0.70 / 0.74 / -5.4%
tpcds_q17/datafusion:parquet 75108042 / 74179220 / +1.3% 101094659 / 99947249 / +1.1% 0.74 / 0.74 / +0.1%
tpcds_q18/datafusion:parquet 68098414 / 68176314 / -0.1% 89500207 / 90601495 / -1.2% 0.76 / 0.75 / +1.1%
tpcds_q19/datafusion:parquet 29742162 / 29979450 / -0.8% 45110937 / 42461266 / +6.2% 0.66 / 0.71 / -6.6%
tpcds_q20/datafusion:parquet 25589344 / 25299084 / +1.1% 38451801 / 38763993 / -0.8% 0.67 / 0.65 / +2.0%
tpcds_q21/datafusion:parquet 20337342 / 20143853 / +1.0% 35128823 / 32533897 / +8.0% 0.58 / 0.62 / -6.5%
tpcds_q22/datafusion:parquet 77794850 / 77992354 / -0.3% 90025940 / 88630266 / +1.6% 0.86 / 0.88 / -1.8%
tpcds_q23/datafusion:parquet 93023984 / 92070006 / +1.0% 114163466 / 115685844 / -1.3% 0.81 / 0.80 / +2.4%
tpcds_q24/datafusion:parquet 91595930 / 91320906 / +0.3% 117584347 / 114776797 / +2.4% 0.78 / 0.80 / -2.1%
tpcds_q25/datafusion:parquet 82076244 / 80147062 / +2.4% 101452278 / 100467329 / +1.0% 0.81 / 0.80 / +1.4%
tpcds_q26/datafusion:parquet 44253828 / 43368406 / +2.0% 60192418 / 62993503 / -4.4% 0.74 / 0.69 / +6.8%
tpcds_q27/datafusion:parquet 77454730 / 77766271 / -0.4% 101787007 / 95385028 / +6.7% 0.76 / 0.82 / -6.7%
tpcds_q28/datafusion:parquet 28758047 / 28898218 / -0.5% 44728467 / 41658255 / +7.4% 0.64 / 0.69 / -7.3%
tpcds_q29/datafusion:parquet 76327234 / 74946869 / +1.8% 100474517 / 95881548 / +4.8% 0.76 / 0.78 / -2.8%
tpcds_q30/datafusion:parquet 45513246 / 43893362 / +3.7% 58165709 / 59749504 / -2.7% 0.78 / 0.73 / +6.5%
tpcds_q31/datafusion:parquet 82172682 / 81440329 / +0.9% 99773243 / 100055168 / -0.3% 0.82 / 0.81 / +1.2%
tpcds_q32/datafusion:parquet 26618149 / 27143890 / -1.9% 41723213 / 40303899 / +3.5% 0.64 / 0.67 / -5.3%
tpcds_q33/datafusion:parquet 38434635 / 37398043 / +2.8% 54903889 / 54795235 / +0.2% 0.70 / 0.68 / +2.6%
tpcds_q34/datafusion:parquet 24615656 / 24635061 / -0.1% 37322557 / 36342546 / +2.7% 0.66 / 0.68 / -2.7%
tpcds_q35/datafusion:parquet 50990433 / 50802148 / +0.4% 64059699 / 64573709 / -0.8% 0.80 / 0.79 / +1.2%
tpcds_q36/datafusion:parquet 52223002 / 52685074 / -0.9% 73449568 / 74511270 / -1.4% 0.71 / 0.71 / +0.6%
tpcds_q37/datafusion:parquet 21124011 / 21130209 / -0.0% 34387442 / 32955418 / +4.3% 0.61 / 0.64 / -4.2%
tpcds_q38/datafusion:parquet 45586362 / 45074440 / +1.1% 63822631 / 61169495 / +4.3% 0.71 / 0.74 / -3.1%
tpcds_q39/datafusion:parquet 68403638 / 67143299 / +1.9% 84520150 / 87798285 / -3.7% 0.81 / 0.76 / +5.8%
tpcds_q40/datafusion:parquet 30385866 / 31732224 / -4.2% 51190396 / 51366919 / -0.3% 0.59 / 0.62 / -3.9%
tpcds_q41/datafusion:parquet 20303604 / 19560677 / +3.8% 33163537 / 30927269 / +7.2% 0.61 / 0.63 / -3.2%
tpcds_q42/datafusion:parquet 18618472 / 18303020 / +1.7% 32622168 / 30195041 / +8.0% 0.57 / 0.61 / -5.8%
tpcds_q43/datafusion:parquet 19617842 / 19552828 / +0.3% 32274090 / 32452403 / -0.5% 0.61 / 0.60 / +0.9%
tpcds_q44/datafusion:parquet 35507610 / 35238887 / +0.8% 51229326 / 50018816 / +2.4% 0.69 / 0.70 / -1.6%
tpcds_q45/datafusion:parquet 26857993 / 27571686 / -2.6% 39976135 / 42507478 / -6.0% 0.67 / 0.65 / +3.6%
tpcds_q46/datafusion:parquet 29557620 / 29861832 / -1.0% 47075523 / 46926615 / +0.3% 0.63 / 0.64 / -1.3%
tpcds_q47/datafusion:parquet 99609662 / 98146604 / +1.5% 122937995 / 122067336 / +0.7% 0.81 / 0.80 / +0.8%
tpcds_q48/datafusion:parquet 43095076 / 42447005 / +1.5% 58392903 / 59145720 / -1.3% 0.74 / 0.72 / +2.8%
tpcds_q49/datafusion:parquet 71106934 / 72220930 / -1.5% 100075556 / 96046791 / +4.2% 0.71 / 0.75 / -5.5%
tpcds_q50/datafusion:parquet 49169283 / 47512774 / +3.5% 66198994 / 71800733 / -7.8% 0.74 / 0.66 / +12.2%
tpcds_q51/datafusion:parquet 75502978 / 75603896 / -0.1% 95583999 / 96356391 / -0.8% 0.79 / 0.78 / +0.7%
tpcds_q52/datafusion:parquet 18843902 / 18667056 / +0.9% 30807482 / 29454574 / +4.6% 0.61 / 0.63 / -3.5%
tpcds_q53/datafusion:parquet 22657750 / 23176485 / -2.2% 39020374 / 39182736 / -0.4% 0.58 / 0.59 / -1.8%
tpcds_q54/datafusion:parquet 53779917 / 51885012 / +3.7% 69341330 / 69416818 / -0.1% 0.78 / 0.75 / +3.8%
tpcds_q55/datafusion:parquet 18441440 / 18301546 / +0.8% 28851786 / 31877199 / -9.5% 0.64 / 0.57 / +11.3%
tpcds_q56/datafusion:parquet 42181952 / 39632970 / +6.4% 54744236 / 55628203 / -1.6% 0.77 / 0.71 / +8.2%
tpcds_q57/datafusion:parquet 102758550 / 100454820 / +2.3% 115416731 / 121808907 / -5.2% 0.89 / 0.82 / +8.0%
tpcds_q58/datafusion:parquet 64903329 / 65082382 / -0.3% 82891282 / 84884800 / -2.3% 0.78 / 0.77 / +2.1%
tpcds_q59/datafusion:parquet 49774707 / 47924398 / +3.9% 66205043 / 68304504 / -3.1% 0.75 / 0.70 / +7.2%
tpcds_q60/datafusion:parquet 40617036 / 42313918 / -4.0% 55840759 / 60333944 / -7.4% 0.73 / 0.70 / +3.7%
tpcds_q61/datafusion:parquet 36618715 / 36404834 / +0.6% 51857985 / 50470158 / +2.7% 0.71 / 0.72 / -2.1%
tpcds_q62/datafusion:parquet 25471098 / 25240186 / +0.9% 38396908 / 37930554 / +1.2% 0.66 / 0.67 / -0.3%
tpcds_q63/datafusion:parquet 22552465 / 22274464 / +1.2% 38985495 / 40137287 / -2.9% 0.58 / 0.55 / +4.2%
tpcds_q64/datafusion:parquet 288246001 / 291825881 / -1.2% 327724000 / 326894100 / +0.3% 0.88 / 0.89 / -1.5%
tpcds_q65/datafusion:parquet 123915857 / 121713454 / +1.8% 141778174 / 142669978 / -0.6% 0.87 / 0.85 / +2.4%
tpcds_q66/datafusion:parquet 62786862 / 59440828 / +5.6% 77831777 / 75640564 / +2.9% 0.81 / 0.79 / +2.7%
tpcds_q67/datafusion:parquet 68843086 / 68348957 / +0.7% 92595892 / 93585303 / -1.1% 0.74 / 0.73 / +1.8%
tpcds_q68/datafusion:parquet 29579187 / 29525206 / +0.2% 49251731 / 49365427 / -0.2% 0.60 / 0.60 / +0.4%
tpcds_q69/datafusion:parquet 44269120 / 42326063 / +4.6% 60488215 / 56487138 / +7.1% 0.73 / 0.75 / -2.3%
tpcds_q70/datafusion:parquet 81340448 / 80186448 / +1.4% 97566947 / 98784851 / -1.2% 0.83 / 0.81 / +2.7%
tpcds_q71/datafusion:parquet 27574778 / 26321141 / +4.8% 38355257 / 36794416 / +4.2% 0.72 / 0.72 / +0.5%
tpcds_q72/datafusion:parquet 525934250 / 524180635 / +0.3% 551579101 / 558063742 / -1.2% 0.95 / 0.94 / +1.5%
tpcds_q73/datafusion:parquet 24010847 / 24084504 / -0.3% 37424595 / 36840445 / +1.6% 0.64 / 0.65 / -1.9%
tpcds_q74/datafusion:parquet 70820792 / 70174123 / +0.9% 89544113 / 90258255 / -0.8% 0.79 / 0.78 / +1.7%
tpcds_q75/datafusion:parquet 155545210 / 156896393 / -0.9% 184135866 / 184038891 / +0.1% 0.84 / 0.85 / -0.9%
tpcds_q76/datafusion:parquet 33776159 / 34224952 / -1.3% 48321452 / 50692625 / -4.7% 0.70 / 0.68 / +3.5%
tpcds_q77/datafusion:parquet 57633500 / 58233668 / -1.0% 75216641 / 76077433 / -1.1% 0.77 / 0.77 / +0.1%
tpcds_q78/datafusion:parquet 88016986 / 85428305 / +3.0% 118979710 / 118447829 / +0.4% 0.74 / 0.72 / +2.6%
tpcds_q79/datafusion:parquet 27664320 / 27134527 / +2.0% 43302479 / 42881579 / +1.0% 0.64 / 0.63 / +1.0%
tpcds_q80/datafusion:parquet 80263826 / 80657915 / -0.5% 115371460 / 110802292 / +4.1% 0.70 / 0.73 / -4.4%
tpcds_q81/datafusion:parquet 36948606 / 37777232 / -2.2% 52715113 / 53177596 / -0.9% 0.70 / 0.71 / -1.3%
tpcds_q82/datafusion:parquet 22735174 / 21830275 / +4.1% 35504801 / 35544461 / -0.1% 0.64 / 0.61 / +4.3%
tpcds_q83/datafusion:parquet 59230672 / 58296937 / +1.6% 77228690 / 74616736 / +3.5% 0.77 / 0.78 / -1.8%
tpcds_q84/datafusion:parquet 27122546 / 26330226 / +3.0% 41237568 / 42384381 / -2.7% 0.66 / 0.62 / +5.9%
tpcds_q85/datafusion:parquet 80500800 / 81149270 / -0.8% 104378518 / 105837872 / -1.4% 0.77 / 0.77 / +0.6%
tpcds_q86/datafusion:parquet 21132144 / 21226134 / -0.4% 34362430 / 33972858 / +1.1% 0.61 / 0.62 / -1.6%
tpcds_q87/datafusion:parquet 45844048 / 43655820 / +5.0% 65468995 / 61819470 / +5.9% 0.70 / 0.71 / -0.8%
tpcds_q88/datafusion:parquet 37162379 / 37317510 / -0.4% 51463158 / 52037306 / -1.1% 0.72 / 0.72 / +0.7%
tpcds_q89/datafusion:parquet 26198665 / 25620515 / +2.3% 42635487 / 43168659 / -1.2% 0.61 / 0.59 / +3.5%
tpcds_q90/datafusion:parquet 15129718 / 14700265 / +2.9% 26953484 / 26744264 / +0.8% 0.56 / 0.55 / +2.1%
tpcds_q91/datafusion:parquet 32619824 / 32536375 / +0.3% 46781105 / 46834523 / -0.1% 0.70 / 0.69 / +0.4%
tpcds_q92/datafusion:parquet 27362885 / 26925968 / +1.6% 40851264 / 41562779 / -1.7% 0.67 / 0.65 / +3.4%
tpcds_q93/datafusion:parquet 27408271 / 27737618 / -1.2% 48177799 / 47560036 / +1.3% 0.57 / 0.58 / -2.5%
tpcds_q94/datafusion:parquet 36095563 / 34987990 / +3.2% 49441664 / 49838089 / -0.8% 0.73 / 0.70 / +4.0%
tpcds_q95/datafusion:parquet 64626412 / 63986628 / +1.0% 78588488 / 82628090 / -4.9% 0.82 / 0.77 / +6.2%
tpcds_q96/datafusion:parquet 12721578 / 12515555 / +1.6% 24925842 / 24364474 / +2.3% 0.51 / 0.51 / -0.6%
tpcds_q97/datafusion:parquet 36432156 / 35645164 / +2.2% 51342584 / 49168462 / +4.4% 0.71 / 0.72 / -2.1%
tpcds_q98/datafusion:parquet 25607947 / 24769713 / +3.4% 38480994 / 39883907 / -3.5% 0.67 / 0.62 / +7.2%
tpcds_q99/datafusion:parquet 30203996 / 29456394 / +2.5% 42016941 / 42511842 / -1.2% 0.72 / 0.69 / +3.7%
duckdb / vortex-file-compressed / ns (1.001x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/duckdb:vortex-file-compressed 16465950 / 16444475 / +0.1% 25202934 / 25468340 / -1.0% 0.65 / 0.65 / +1.2%
tpcds_q02/duckdb:vortex-file-compressed 16329180 / 16460483 / -0.8% 22756003 / 23822245 / -4.5% 0.72 / 0.69 / +3.9%
tpcds_q03/duckdb:vortex-file-compressed 7095278 / 7076496 / +0.3% 16364921 / 16811089 / -2.7% 0.43 / 0.42 / +3.0%
tpcds_q04/duckdb:vortex-file-compressed 46026191 / 45671773 / +0.8% 56134715 / 56795457 / -1.2% 0.82 / 0.80 / +2.0%
tpcds_q05/duckdb:vortex-file-compressed 23261580 / 23444196 / -0.8% 33230030 / 33922893 / -2.0% 0.70 / 0.69 / +1.3%
tpcds_q06/duckdb:vortex-file-compressed 24441105 / 24544822 / -0.4% 39176196 / 39101350 / +0.2% 0.62 / 0.63 / -0.6%
tpcds_q07/duckdb:vortex-file-compressed 13485015 / 13574572 / -0.7% 24663548 / 23853354 / +3.4% 0.55 / 0.57 / -3.9%
tpcds_q08/duckdb:vortex-file-compressed 17522183 / 17694292 / -1.0% 26162455 / 27427836 / -4.6% 0.67 / 0.65 / +3.8%
tpcds_q09/duckdb:vortex-file-compressed 19157832 / 19230780 / -0.4% 30509258 / 29232867 / +4.4% 0.63 / 0.66 / -4.5%
tpcds_q10/duckdb:vortex-file-compressed 28001198 / 28127254 / -0.4% 42523256 / 41633782 / +2.1% 0.66 / 0.68 / -2.5%
tpcds_q11/duckdb:vortex-file-compressed 37043376 / 36907482 / +0.4% 46881382 / 45373610 / +3.3% 0.79 / 0.81 / -2.9%
tpcds_q12/duckdb:vortex-file-compressed 10207990 / 10175710 / +0.3% 17177536 / 17230057 / -0.3% 0.59 / 0.59 / +0.6%
tpcds_q13/duckdb:vortex-file-compressed 20995472 / 21038962 / -0.2% 32114581 / 34714581 / -7.5% 0.65 / 0.61 / +7.9%
tpcds_q14/duckdb:vortex-file-compressed 57389334 / 57724936 / -0.6% 68484072 / 68632960 / -0.2% 0.84 / 0.84 / -0.4%
tpcds_q15/duckdb:vortex-file-compressed 11362726 / 11248794 / +1.0% 18771846 / 18537256 / +1.3% 0.61 / 0.61 / -0.2%
tpcds_q16/duckdb:vortex-file-compressed 24421840 / 24567762 / -0.6% 35960544 / 35296980 / +1.9% 0.68 / 0.70 / -2.4%
tpcds_q17/duckdb:vortex-file-compressed 24584474 / 24773062 / -0.8% 37865691 / 36689010 / +3.2% 0.65 / 0.68 / -3.8%
tpcds_q18/duckdb:vortex-file-compressed 32033260 / 31704231 / +1.0% 41882479 / 41795055 / +0.2% 0.76 / 0.76 / +0.8%
tpcds_q19/duckdb:vortex-file-compressed 78464352 / 78453346 / +0.0% 89141273 / 88861136 / +0.3% 0.88 / 0.88 / -0.3%
tpcds_q20/duckdb:vortex-file-compressed 11246590 / 11302434 / -0.5% 18215521 / 17921560 / +1.6% 0.62 / 0.63 / -2.1%
tpcds_q21/duckdb:vortex-file-compressed 10139355 / 9613981 / +5.5% 22142047 / 21618546 / +2.4% 0.46 / 0.44 / +3.0%
tpcds_q22/duckdb:vortex-file-compressed 17822220 / 17475264 / +2.0% 28786694 / 29801515 / -3.4% 0.62 / 0.59 / +5.6%
tpcds_q23/duckdb:vortex-file-compressed 38591476 / 38469427 / +0.3% 50469093 / 50493626 / -0.0% 0.76 / 0.76 / +0.4%
tpcds_q24/duckdb:vortex-file-compressed 26889983 / 27034658 / -0.5% 37021812 / 38258066 / -3.2% 0.73 / 0.71 / +2.8%
tpcds_q25/duckdb:vortex-file-compressed 23523593 / 23233486 / +1.2% 36349958 / 35619880 / +2.0% 0.65 / 0.65 / -0.8%
tpcds_q26/duckdb:vortex-file-compressed 12915936 / 13489824 / -4.3% 20914557 / 21359876 / -2.1% 0.62 / 0.63 / -2.2%
tpcds_q27/duckdb:vortex-file-compressed 16239698 / 16130987 / +0.7% 26842951 / 27091120 / -0.9% 0.60 / 0.60 / +1.6%
tpcds_q28/duckdb:vortex-file-compressed 15057760 / 15571143 / -3.3% 26035830 / 26383560 / -1.3% 0.58 / 0.59 / -2.0%
tpcds_q29/duckdb:vortex-file-compressed 22741156 / 23097506 / -1.5% 36264433 / 35788503 / +1.3% 0.63 / 0.65 / -2.8%
tpcds_q30/duckdb:vortex-file-compressed 16203065 / 16286234 / -0.5% 24685627 / 24987373 / -1.2% 0.66 / 0.65 / +0.7%
tpcds_q31/duckdb:vortex-file-compressed 18142228 / 18038853 / +0.6% 26148815 / 26835741 / -2.6% 0.69 / 0.67 / +3.2%
tpcds_q32/duckdb:vortex-file-compressed 12166195 / 12118126 / +0.4% 21881352 / 21035329 / +4.0% 0.56 / 0.58 / -3.5%
tpcds_q33/duckdb:vortex-file-compressed 18835869 / 19145275 / -1.6% 28565794 / 28982771 / -1.4% 0.66 / 0.66 / -0.2%
tpcds_q34/duckdb:vortex-file-compressed 13100650 / 13291362 / -1.4% 22560194 / 23346380 / -3.4% 0.58 / 0.57 / +2.0%
tpcds_q35/duckdb:vortex-file-compressed 28700488 / 29057151 / -1.2% 40464579 / 41060035 / -1.5% 0.71 / 0.71 / +0.2%
tpcds_q36/duckdb:vortex-file-compressed 14822782 / 14673114 / +1.0% 25188088 / 25300723 / -0.4% 0.59 / 0.58 / +1.5%
tpcds_q37/duckdb:vortex-file-compressed 9753081 / 9811254 / -0.6% 24828934 / 24475303 / +1.4% 0.39 / 0.40 / -2.0%
tpcds_q38/duckdb:vortex-file-compressed 24643354 / 24605265 / +0.2% 32338078 / 32323829 / +0.0% 0.76 / 0.76 / +0.1%
tpcds_q39/duckdb:vortex-file-compressed 19028718 / 18754580 / +1.5% 29713474 / 31312343 / -5.1% 0.64 / 0.60 / +6.9%
tpcds_q40/duckdb:vortex-file-compressed 13974222 / 14358350 / -2.7% 20755855 / 21137017 / -1.8% 0.67 / 0.68 / -0.9%
tpcds_q41/duckdb:vortex-file-compressed 19001006 / 19083947 / -0.4% 26239698 / 26661569 / -1.6% 0.72 / 0.72 / +1.2%
tpcds_q42/duckdb:vortex-file-compressed 7233490 / 7055646 / +2.5% 16152465 / 15785619 / +2.3% 0.45 / 0.45 / +0.2%
tpcds_q43/duckdb:vortex-file-compressed 10629382 / 10468877 / +1.5% 19819946 / 20047716 / -1.1% 0.54 / 0.52 / +2.7%
tpcds_q44/duckdb:vortex-file-compressed 12788688 / 12765054 / +0.2% 24572098 / 24035788 / +2.2% 0.52 / 0.53 / -2.0%
tpcds_q45/duckdb:vortex-file-compressed 23752470 / 24095519 / -1.4% 31763760 / 31439773 / +1.0% 0.75 / 0.77 / -2.4%
tpcds_q46/duckdb:vortex-file-compressed 19806934 / 19944214 / -0.7% 28199232 / 28498573 / -1.1% 0.70 / 0.70 / +0.4%
tpcds_q47/duckdb:vortex-file-compressed 35029860 / 34362920 / +1.9% 44327593 / 45200024 / -1.9% 0.79 / 0.76 / +3.9%
tpcds_q48/duckdb:vortex-file-compressed 23475132 / 23537748 / -0.3% 35646000 / 35171768 / +1.3% 0.66 / 0.67 / -1.6%
tpcds_q49/duckdb:vortex-file-compressed 19599216 / 20061539 / -2.3% 31861711 / 31020788 / +2.7% 0.62 / 0.65 / -4.9%
tpcds_q50/duckdb:vortex-file-compressed 17511986 / 17224026 / +1.7% 27830116 / 28249732 / -1.5% 0.63 / 0.61 / +3.2%
tpcds_q51/duckdb:vortex-file-compressed 49136188 / 49558842 / -0.9% 60224873 / 59142497 / +1.8% 0.82 / 0.84 / -2.6%
tpcds_q52/duckdb:vortex-file-compressed 7272364 / 7471124 / -2.7% 16130403 / 16155821 / -0.2% 0.45 / 0.46 / -2.5%
tpcds_q53/duckdb:vortex-file-compressed 11616848 / 11708613 / -0.8% 21394396 / 20430202 / +4.7% 0.54 / 0.57 / -5.3%
tpcds_q54/duckdb:vortex-file-compressed 20630199 / 20517178 / +0.6% 31012101 / 30341791 / +2.2% 0.67 / 0.68 / -1.6%
tpcds_q55/duckdb:vortex-file-compressed 7163716 / 7086945 / +1.1% 15987528 / 16614933 / -3.8% 0.45 / 0.43 / +5.1%
tpcds_q56/duckdb:vortex-file-compressed 18219078 / 18390555 / -0.9% 27793261 / 28690113 / -3.1% 0.66 / 0.64 / +2.3%
tpcds_q57/duckdb:vortex-file-compressed 26140523 / 26501728 / -1.4% 34341984 / 35075063 / -2.1% 0.76 / 0.76 / +0.7%
tpcds_q58/duckdb:vortex-file-compressed 20919890 / 20471580 / +2.2% 30876370 / 30978872 / -0.3% 0.68 / 0.66 / +2.5%
tpcds_q59/duckdb:vortex-file-compressed 20171260 / 20144962 / +0.1% 27530209 / 28840927 / -4.5% 0.73 / 0.70 / +4.9%
tpcds_q60/duckdb:vortex-file-compressed 19090776 / 18663288 / +2.3% 28722964 / 29105972 / -1.3% 0.66 / 0.64 / +3.7%
tpcds_q61/duckdb:vortex-file-compressed 25287795 / 25454070 / -0.7% 34394053 / 35467454 / -3.0% 0.74 / 0.72 / +2.4%
tpcds_q62/duckdb:vortex-file-compressed 12101670 / 12066966 / +0.3% 18941877 / 19142635 / -1.0% 0.64 / 0.63 / +1.4%
tpcds_q63/duckdb:vortex-file-compressed 11243334 / 11315712 / -0.6% 20436136 / 20240474 / +1.0% 0.55 / 0.56 / -1.6%
tpcds_q64/duckdb:vortex-file-compressed 50556544 / 50298476 / +0.5% 66735023 / 68843128 / -3.1% 0.76 / 0.73 / +3.7%
tpcds_q65/duckdb:vortex-file-compressed 13225997 / 13261522 / -0.3% 24256091 / 23640025 / +2.6% 0.55 / 0.56 / -2.8%
tpcds_q66/duckdb:vortex-file-compressed 30063332 / 29356364 / +2.4% 40627418 / 38953267 / +4.3% 0.74 / 0.75 / -1.8%
tpcds_q67/duckdb:vortex-file-compressed 58792832 / 58699637 / +0.2% 67220437 / 67480625 / -0.4% 0.87 / 0.87 / +0.5%
tpcds_q68/duckdb:vortex-file-compressed 19650402 / 19803436 / -0.8% 28099388 / 28546143 / -1.6% 0.70 / 0.69 / +0.8%
tpcds_q69/duckdb:vortex-file-compressed 23173908 / 23406440 / -1.0% 40097082 / 38019199 / +5.5% 0.58 / 0.62 / -6.1%
tpcds_q70/duckdb:vortex-file-compressed 16323976 / 16057534 / +1.7% 27438740 / 28434674 / -3.5% 0.59 / 0.56 / +5.3%
tpcds_q71/duckdb:vortex-file-compressed 13748271 / 13083326 / +5.1% 23912176 / 24266057 / -1.5% 0.57 / 0.54 / +6.6%
tpcds_q72/duckdb:vortex-file-compressed 53788462 / 53306598 / +0.9% 66057725 / 66841585 / -1.2% 0.81 / 0.80 / +2.1%
tpcds_q73/duckdb:vortex-file-compressed 11869728 / 12089608 / -1.8% 20834978 / 21036740 / -1.0% 0.57 / 0.57 / -0.9%
tpcds_q74/duckdb:vortex-file-compressed 26214334 / 26070268 / +0.6% 34621188 / 34365954 / +0.7% 0.76 / 0.76 / -0.2%
tpcds_q75/duckdb:vortex-file-compressed 28231066 / 28699944 / -1.6% 40076861 / 39734586 / +0.9% 0.70 / 0.72 / -2.5%
tpcds_q76/duckdb:vortex-file-compressed 13187149 / 13454318 / -2.0% 25479717 / 25508048 / -0.1% 0.52 / 0.53 / -1.9%
tpcds_q77/duckdb:vortex-file-compressed 21434210 / 21028818 / +1.9% 33372931 / 32628785 / +2.3% 0.64 / 0.64 / -0.3%
tpcds_q78/duckdb:vortex-file-compressed 43739512 / 43216690 / +1.2% 55909482 / 54126335 / +3.3% 0.78 / 0.80 / -2.0%
tpcds_q79/duckdb:vortex-file-compressed 13629315 / 13435902 / +1.4% 23038156 / 23432828 / -1.7% 0.59 / 0.57 / +3.2%
tpcds_q80/duckdb:vortex-file-compressed 37814160 / 37202952 / +1.6% 51041657 / 51588198 / -1.1% 0.74 / 0.72 / +2.7%
tpcds_q81/duckdb:vortex-file-compressed 22098954 / 21286061 / +3.8% 30213306 / 30091464 / +0.4% 0.73 / 0.71 / +3.4%
tpcds_q82/duckdb:vortex-file-compressed 9748048 / 9434022 / +3.3% 25418233 / 25789878 / -1.4% 0.38 / 0.37 / +4.8%
tpcds_q83/duckdb:vortex-file-compressed 17806444 / 18050168 / -1.4% 25976217 / 24595920 / +5.6% 0.69 / 0.73 / -6.6%
tpcds_q84/duckdb:vortex-file-compressed 15763846 / 15788435 / -0.2% 23571001 / 23186803 / +1.7% 0.67 / 0.68 / -1.8%
tpcds_q85/duckdb:vortex-file-compressed 24816525 / 24704460 / +0.5% 36984344 / 36707520 / +0.8% 0.67 / 0.67 / -0.3%
tpcds_q86/duckdb:vortex-file-compressed 9875848 / 9966084 / -0.9% 17105163 / 16937119 / +1.0% 0.58 / 0.59 / -1.9%
tpcds_q87/duckdb:vortex-file-compressed 26251290 / 26271254 / -0.1% 33947686 / 33892892 / +0.2% 0.77 / 0.78 / -0.2%
tpcds_q88/duckdb:vortex-file-compressed 31239966 / 31419104 / -0.6% 40237522 / 39845592 / +1.0% 0.78 / 0.79 / -1.5%
tpcds_q89/duckdb:vortex-file-compressed 12426230 / 12523714 / -0.8% 22450194 / 22366688 / +0.4% 0.55 / 0.56 / -1.1%
tpcds_q90/duckdb:vortex-file-compressed 8448780 / 8234150 / +2.6% 16046866 / 16219806 / -1.1% 0.53 / 0.51 / +3.7%
tpcds_q91/duckdb:vortex-file-compressed 14664983 / 14875414 / -1.4% 22514140 / 23432472 / -3.9% 0.65 / 0.63 / +2.6%
tpcds_q92/duckdb:vortex-file-compressed 12464940 / 12185658 / +2.3% 20962219 / 21289808 / -1.5% 0.59 / 0.57 / +3.9%
tpcds_q93/duckdb:vortex-file-compressed 10492934 / 10200146 / +2.9% 24010643 / 24307062 / -1.2% 0.44 / 0.42 / +4.1%
tpcds_q94/duckdb:vortex-file-compressed 22126588 / 22700503 / -2.5% 32663910 / 32522349 / +0.4% 0.68 / 0.70 / -3.0%
tpcds_q95/duckdb:vortex-file-compressed 63021374 / 63017334 / +0.0% 72997194 / 71429073 / +2.2% 0.86 / 0.88 / -2.1%
tpcds_q96/duckdb:vortex-file-compressed 7092641 / 7044906 / +0.7% 15803831 / 16074625 / -1.7% 0.45 / 0.44 / +2.4%
tpcds_q97/duckdb:vortex-file-compressed 23471561 / 23644552 / -0.7% 31388904 / 32299442 / -2.8% 0.75 / 0.73 / +2.1%
tpcds_q98/duckdb:vortex-file-compressed 11767317 / 11336208 / +3.8% 19813054 / 21107734 / -6.1% 0.59 / 0.54 / +10.6%
tpcds_q99/duckdb:vortex-file-compressed 15106018 / 15282436 / -1.2% 23235763 / 24087996 / -3.5% 0.65 / 0.63 / +2.5%
duckdb / parquet / ns (1.006x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/duckdb:parquet 24340312 / 24461415 / -0.5% 31418210 / 31889119 / -1.5% 0.77 / 0.77 / +1.0%
tpcds_q02/duckdb:parquet 18253618 / 18477803 / -1.2% 24688515 / 24904542 / -0.9% 0.74 / 0.74 / -0.3%
tpcds_q03/duckdb:parquet 9776401 / 9885219 / -1.1% 15566294 / 15716141 / -1.0% 0.63 / 0.63 / -0.1%
tpcds_q04/duckdb:parquet 118664630 / 118340878 / +0.3% 126674216 / 127118353 / -0.3% 0.94 / 0.93 / +0.6%
tpcds_q05/duckdb:parquet 29082635 / 28904132 / +0.6% 34022934 / 34074757 / -0.2% 0.85 / 0.85 / +0.8%
tpcds_q06/duckdb:parquet 26660374 / 26612744 / +0.2% 36549112 / 36038010 / +1.4% 0.73 / 0.74 / -1.2%
tpcds_q07/duckdb:parquet 31486910 / 30889214 / +1.9% 39067373 / 39478001 / -1.0% 0.81 / 0.78 / +3.0%
tpcds_q08/duckdb:parquet 20499002 / 20187230 / +1.5% 26778212 / 26525878 / +1.0% 0.77 / 0.76 / +0.6%
tpcds_q09/duckdb:parquet 27446514 / 27513464 / -0.2% 34816846 / 31524831 / +10.4% 🔴 0.79 / 0.87 / -9.7%
tpcds_q10/duckdb:parquet 33400530 / 33222414 / +0.5% 41536960 / 40837713 / +1.7% 0.80 / 0.81 / -1.2%
tpcds_q11/duckdb:parquet 68981762 / 68731161 / +0.4% 77036423 / 76879564 / +0.2% 0.90 / 0.89 / +0.2%
tpcds_q12/duckdb:parquet 14298996 / 14144158 / +1.1% 19352201 / 19819277 / -2.4% 0.74 / 0.71 / +3.5%
tpcds_q13/duckdb:parquet 31876686 / 31292196 / +1.9% 40594641 / 40289029 / +0.8% 0.79 / 0.78 / +1.1%
tpcds_q14/duckdb:parquet 65050494 / 64338646 / +1.1% 72111326 / 71335260 / +1.1% 0.90 / 0.90 / +0.0%
tpcds_q15/duckdb:parquet 16899958 / 17339696 / -2.5% 23186349 / 23248232 / -0.3% 0.73 / 0.75 / -2.3%
tpcds_q16/duckdb:parquet 30754776 / 30578287 / +0.6% 37768055 / 37670788 / +0.3% 0.81 / 0.81 / +0.3%
tpcds_q17/duckdb:parquet 35121418 / 35393214 / -0.8% 41244066 / 41074208 / +0.4% 0.85 / 0.86 / -1.2%
tpcds_q18/duckdb:parquet 48158688 / 47513666 / +1.4% 56270951 / 55915480 / +0.6% 0.86 / 0.85 / +0.7%
tpcds_q19/duckdb:parquet 19457738 / 18908272 / +2.9% 26285305 / 25869161 / +1.6% 0.74 / 0.73 / +1.3%
tpcds_q20/duckdb:parquet 15195292 / 15117726 / +0.5% 20255156 / 20319039 / -0.3% 0.75 / 0.74 / +0.8%
tpcds_q21/duckdb:parquet 10785268 / 10958786 / -1.6% 17677990 / 16915770 / +4.5% 0.61 / 0.65 / -5.8%
tpcds_q22/duckdb:parquet 20984520 / 21167452 / -0.9% 27040678 / 27090109 / -0.2% 0.78 / 0.78 / -0.7%
tpcds_q23/duckdb:parquet 43777052 / 42076472 / +4.0% 48651244 / 50375183 / -3.4% 0.90 / 0.84 / +7.7%
tpcds_q24/duckdb:parquet 32557509 / 30720912 / +6.0% 37596907 / 36885762 / +1.9% 0.87 / 0.83 / +4.0%
tpcds_q25/duckdb:parquet 31067860 / 30979612 / +0.3% 37830533 / 38374618 / -1.4% 0.82 / 0.81 / +1.7%
tpcds_q26/duckdb:parquet 29284462 / 29260556 / +0.1% 36071948 / 35609138 / +1.3% 0.81 / 0.82 / -1.2%
tpcds_q27/duckdb:parquet 34202584 / 33134471 / +3.2% 43224713 / 43669624 / -1.0% 0.79 / 0.76 / +4.3%
tpcds_q28/duckdb:parquet 26731336 / 26638018 / +0.4% 33954020 / 32338357 / +5.0% 0.79 / 0.82 / -4.4%
tpcds_q29/duckdb:parquet 32646378 / 32629098 / +0.1% 40086531 / 40449665 / -0.9% 0.81 / 0.81 / +1.0%
tpcds_q30/duckdb:parquet 36077188 / 36216204 / -0.4% 45525545 / 44941640 / +1.3% 0.79 / 0.81 / -1.7%
tpcds_q31/duckdb:parquet 35976207 / 36054844 / -0.2% 43427592 / 43183235 / +0.6% 0.83 / 0.83 / -0.8%
tpcds_q32/duckdb:parquet 14543972 / 14508078 / +0.2% 21443471 / 21278004 / +0.8% 0.68 / 0.68 / -0.5%
tpcds_q33/duckdb:parquet 23809762 / 23632444 / +0.8% 29596583 / 29707542 / -0.4% 0.80 / 0.80 / +1.1%
tpcds_q34/duckdb:parquet 17225818 / 17264167 / -0.2% 24513625 / 24340721 / +0.7% 0.70 / 0.71 / -0.9%
tpcds_q35/duckdb:parquet 35561682 / 35442622 / +0.3% 44635039 / 44681249 / -0.1% 0.80 / 0.79 / +0.4%
tpcds_q36/duckdb:parquet 41751108 / 40503322 / +3.1% 48135223 / 48119798 / +0.0% 0.87 / 0.84 / +3.0%
tpcds_q37/duckdb:parquet 12994929 / 13397750 / -3.0% 19627769 / 20708401 / -5.2% 0.66 / 0.65 / +2.3%
tpcds_q38/duckdb:parquet 30078450 / 29717562 / +1.2% 34348385 / 34213448 / +0.4% 0.88 / 0.87 / +0.8%
tpcds_q39/duckdb:parquet 23676404 / 23829380 / -0.6% 30626332 / 30449128 / +0.6% 0.77 / 0.78 / -1.2%
tpcds_q40/duckdb:parquet 16428543 / 16238836 / +1.2% 21884173 / 22101711 / -1.0% 0.75 / 0.73 / +2.2%
tpcds_q41/duckdb:parquet 12370754 / 12288706 / +0.7% 18727110 / 18367545 / +2.0% 0.66 / 0.67 / -1.3%
tpcds_q42/duckdb:parquet 9739066 / 9703324 / +0.4% 15741748 / 16110618 / -2.3% 0.62 / 0.60 / +2.7%
tpcds_q43/duckdb:parquet 12520818 / 12636840 / -0.9% 18608691 / 19017623 / -2.2% 0.67 / 0.66 / +1.3%
tpcds_q44/duckdb:parquet 20972174 / 20847950 / +0.6% 33366812 / 27075740 / +23.2% 🔴 0.63 / 0.77 / -18.4%
tpcds_q45/duckdb:parquet 21297925 / 21206582 / +0.4% 28373419 / 28532503 / -0.6% 0.75 / 0.74 / +1.0%
tpcds_q46/duckdb:parquet 36163487 / 36380140 / -0.6% 46583968 / 44539173 / +4.6% 0.78 / 0.82 / -5.0%
tpcds_q47/duckdb:parquet 35146978 / 34586680 / +1.6% 41425017 / 42107682 / -1.6% 0.85 / 0.82 / +3.3%
tpcds_q48/duckdb:parquet 29406982 / 30210846 / -2.7% 37227191 / 39878588 / -6.6% 0.79 / 0.76 / +4.3%
tpcds_q49/duckdb:parquet 29573742 / 30182184 / -2.0% 36736159 / 38576647 / -4.8% 0.81 / 0.78 / +2.9%
tpcds_q50/duckdb:parquet 19947068 / 19863609 / +0.4% 26509902 / 25976470 / +2.1% 0.75 / 0.76 / -1.6%
tpcds_q51/duckdb:parquet 53094341 / 52434272 / +1.3% 77204157 / 60969027 / +26.6% 🔴 0.69 / 0.86 / -20.0%
tpcds_q52/duckdb:parquet 9871785 / 9779850 / +0.9% 15769537 / 15736745 / +0.2% 0.63 / 0.62 / +0.7%
tpcds_q53/duckdb:parquet 14137564 / 13933530 / +1.5% 21279926 / 21080535 / +0.9% 0.66 / 0.66 / +0.5%
tpcds_q54/duckdb:parquet 26370734 / 25205726 / +4.6% 32380938 / 32379265 / +0.0% 0.81 / 0.78 / +4.6%
tpcds_q55/duckdb:parquet 9619288 / 9660020 / -0.4% 15435337 / 15334710 / +0.7% 0.62 / 0.63 / -1.1%
tpcds_q56/duckdb:parquet 25471918 / 26182610 / -2.7% 30746504 / 30957666 / -0.7% 0.83 / 0.85 / -2.0%
tpcds_q57/duckdb:parquet 30944162 / 30862125 / +0.3% 36780584 / 36547991 / +0.6% 0.84 / 0.84 / -0.4%
tpcds_q58/duckdb:parquet 24258584 / 24194040 / +0.3% 30940795 / 30112757 / +2.7% 0.78 / 0.80 / -2.4%
tpcds_q59/duckdb:parquet 23324994 / 23510052 / -0.8% 29467654 / 29836645 / -1.2% 0.79 / 0.79 / +0.5%
tpcds_q60/duckdb:parquet 27865854 / 26534511 / +5.0% 32598803 / 32916450 / -1.0% 0.85 / 0.81 / +6.0%
tpcds_q61/duckdb:parquet 25589556 / 24342124 / +5.1% 28998044 / 29132643 / -0.5% 0.88 / 0.84 / +5.6%
tpcds_q62/duckdb:parquet 13213274 / 13237418 / -0.2% 18656430 / 18208809 / +2.5% 0.71 / 0.73 / -2.6%
tpcds_q63/duckdb:parquet 13507404 / 13446420 / +0.5% 20313669 / 20085179 / +1.1% 0.66 / 0.67 / -0.7%
tpcds_q64/duckdb:parquet 72193800 / 71414432 / +1.1% 83100598 / 83765544 / -0.8% 0.87 / 0.85 / +1.9%
tpcds_q65/duckdb:parquet 21169991 / 19600975 / +8.0% 25933836 / 25574005 / +1.4% 0.82 / 0.77 / +6.5%
tpcds_q66/duckdb:parquet 35286438 / 35508789 / -0.6% 42148572 / 42329909 / -0.4% 0.84 / 0.84 / -0.2%
tpcds_q67/duckdb:parquet 62309862 / 64194316 / -2.9% 67891184 / 68576216 / -1.0% 0.92 / 0.94 / -2.0%
tpcds_q68/duckdb:parquet 29875840 / 28915509 / +3.3% 37200479 / 37384155 / -0.5% 0.80 / 0.77 / +3.8%
tpcds_q69/duckdb:parquet 26214932 / 26301866 / -0.3% 35146168 / 34493925 / +1.9% 0.75 / 0.76 / -2.2%
tpcds_q70/duckdb:parquet 19690540 / 19675632 / +0.1% 27758984 / 27386182 / +1.4% 0.71 / 0.72 / -1.3%
tpcds_q71/duckdb:parquet 17611909 / 17448006 / +0.9% 23958602 / 23990855 / -0.1% 0.74 / 0.73 / +1.1%
tpcds_q72/duckdb:parquet 54763671 / 54660460 / +0.2% 62168931 / 62275059 / -0.2% 0.88 / 0.88 / +0.4%
tpcds_q73/duckdb:parquet 16211047 / 15029432 / +7.9% 21896082 / 22152164 / -1.2% 0.74 / 0.68 / +9.1%
tpcds_q74/duckdb:parquet 84904286 / 84385634 / +0.6% 92182934 / 92185818 / -0.0% 0.92 / 0.92 / +0.6%
tpcds_q75/duckdb:parquet 44931930 / 45624206 / -1.5% 52008485 / 51356188 / +1.3% 0.86 / 0.89 / -2.8%
tpcds_q76/duckdb:parquet 18407486 / 18870968 / -2.5% 24342333 / 24356332 / -0.1% 0.76 / 0.77 / -2.4%
tpcds_q77/duckdb:parquet 24885422 / 24975692 / -0.4% 33304991 / 32645863 / +2.0% 0.75 / 0.77 / -2.3%
tpcds_q78/duckdb:parquet 79553984 / 78898743 / +0.8% 85532885 / 87809925 / -2.6% 0.93 / 0.90 / +3.5%
tpcds_q79/duckdb:parquet 24282336 / 22444963 / +8.2% 29480302 / 29362751 / +0.4% 0.82 / 0.76 / +7.8%
tpcds_q80/duckdb:parquet 37594822 / 36329628 / +3.5% 45359217 / 45211720 / +0.3% 0.83 / 0.80 / +3.1%
tpcds_q81/duckdb:parquet 36931711 / 36806600 / +0.3% 44972689 / 45327130 / -0.8% 0.82 / 0.81 / +1.1%
tpcds_q82/duckdb:parquet 13804562 / 13701072 / +0.8% 20707758 / 20843699 / -0.7% 0.67 / 0.66 / +1.4%
tpcds_q83/duckdb:parquet 20490171 / 20663900 / -0.8% 25382120 / 25360106 / +0.1% 0.81 / 0.81 / -0.9%
tpcds_q84/duckdb:parquet 23084054 / 23496624 / -1.8% 29146834 / 28978214 / +0.6% 0.79 / 0.81 / -2.3%
tpcds_q85/duckdb:parquet 32523144 / 32590619 / -0.2% 40201131 / 39990406 / +0.5% 0.81 / 0.81 / -0.7%
tpcds_q86/duckdb:parquet 12096094 / 12013802 / +0.7% 17226522 / 17020865 / +1.2% 0.70 / 0.71 / -0.5%
tpcds_q87/duckdb:parquet 31375628 / 31554158 / -0.6% 37122215 / 35597054 / +4.3% 0.85 / 0.89 / -4.7%
tpcds_q88/duckdb:parquet 39201395 / 39290797 / -0.2% 45467380 / 45373845 / +0.2% 0.86 / 0.87 / -0.4%
tpcds_q89/duckdb:parquet 14576044 / 14373810 / +1.4% 21586504 / 21425400 / +0.8% 0.68 / 0.67 / +0.7%
tpcds_q90/duckdb:parquet 10289626 / 10363688 / -0.7% 15199332 / 15705175 / -3.2% 0.68 / 0.66 / +2.6%
tpcds_q91/duckdb:parquet 21408460 / 21451967 / -0.2% 29113975 / 28956635 / +0.5% 0.74 / 0.74 / -0.7%
tpcds_q92/duckdb:parquet 15902002 / 15754876 / +0.9% 22533715 / 23375593 / -3.6% 0.71 / 0.67 / +4.7%
tpcds_q93/duckdb:parquet 14322555 / 14160048 / +1.1% 20526943 / 20900109 / -1.8% 0.70 / 0.68 / +3.0%
tpcds_q94/duckdb:parquet 25796898 / 25906744 / -0.4% 33798665 / 33051190 / +2.3% 0.76 / 0.78 / -2.6%
tpcds_q95/duckdb:parquet 44087386 / 44138026 / -0.1% 50451687 / 49404315 / +2.1% 0.87 / 0.89 / -2.2%
tpcds_q96/duckdb:parquet 8393352 / 8380358 / +0.2% 14223723 / 14018596 / +1.5% 0.59 / 0.60 / -1.3%
tpcds_q97/duckdb:parquet 27894322 / 27546754 / +1.3% 34551115 / 34328255 / +0.6% 0.81 / 0.80 / +0.6%
tpcds_q98/duckdb:parquet 16914512 / 16864316 / +0.3% 23661356 / 23676181 / -0.1% 0.71 / 0.71 / +0.4%
tpcds_q99/duckdb:parquet 19229483 / 19262517 / -0.2% 25686707 / 25183158 / +2.0% 0.75 / 0.76 / -2.1%

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Clickbench on NVME 📖

Commits: PR 929c02f6 vs base 19dcd2f2
Verdict: No clear signal (low confidence)
Attributed Vortex impact: -0.7%
Engines: DataFusion No clear signal (-0.0%, low confidence) · DuckDB No clear signal (-1.5%, low confidence)
Vortex (geomean): hot 0.996x ➖ · cold 0.956x ➖
Parquet (geomean): hot 1.003x ➖ · cold 0.987x ➖
Shifts: Parquet (control) +0.3% · Median polish -0.3%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.

datafusion / vortex-file-compressed / ns (0.995x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/datafusion:vortex-file-compressed 1227004 / 1222020 / +0.4% 21598221 / 25360366 / -14.8% 🟢 0.06 / 0.05 / +17.9%
clickbench_q01/datafusion:vortex-file-compressed 6335680 / 6291710 / +0.7% 27681080 / 31998400 / -13.5% 🟢 0.23 / 0.20 / +16.4%
clickbench_q02/datafusion:vortex-file-compressed 10708281 / 10743920 / -0.3% 45050990 / 48726078 / -7.5% 0.24 / 0.22 / +7.8%
clickbench_q03/datafusion:vortex-file-compressed 12732363 / 13578266 / -6.2% 54495889 / 60406156 / -9.8% 0.23 / 0.22 / +3.9%
clickbench_q04/datafusion:vortex-file-compressed 58547415 / 58875622 / -0.6% 101307052 / 104057010 / -2.6% 0.58 / 0.57 / +2.1%
clickbench_q05/datafusion:vortex-file-compressed 77341334 / 78364650 / -1.3% 119374888 / 125165765 / -4.6% 0.65 / 0.63 / +3.5%
clickbench_q06/datafusion:vortex-file-compressed 1277497 / 1245802 / +2.5% 20892531 / 24916123 / -16.1% 🟢 0.06 / 0.05 / +22.3%
clickbench_q07/datafusion:vortex-file-compressed 9390040 / 9440024 / -0.5% 33257123 / 36665765 / -9.3% 0.28 / 0.26 / +9.7%
clickbench_q08/datafusion:vortex-file-compressed 79725778 / 79572816 / +0.2% 122831738 / 128596494 / -4.5% 0.65 / 0.62 / +4.9%
clickbench_q09/datafusion:vortex-file-compressed 170176053 / 176945813 / -3.8% 213690773 / 220521609 / -3.1% 0.80 / 0.80 / -0.8%
clickbench_q10/datafusion:vortex-file-compressed 23076197 / 23324689 / -1.1% 68402790 / 75508962 / -9.4% 0.34 / 0.31 / +9.2%
clickbench_q11/datafusion:vortex-file-compressed 25703420 / 26075788 / -1.4% 68563114 / 77361491 / -11.4% 🟢 0.37 / 0.34 / +11.2%
clickbench_q12/datafusion:vortex-file-compressed 72830669 / 72198817 / +0.9% 120873960 / 127734875 / -5.4% 0.60 / 0.57 / +6.6%
clickbench_q13/datafusion:vortex-file-compressed 116479974 / 117074480 / -0.5% 180757270 / 185295836 / -2.4% 0.64 / 0.63 / +2.0%
clickbench_q14/datafusion:vortex-file-compressed 77602784 / 76569906 / +1.3% 124613020 / 133017726 / -6.3% 0.62 / 0.58 / +8.2%
clickbench_q15/datafusion:vortex-file-compressed 64729284 / 63096343 / +2.6% 95738626 / 103450927 / -7.5% 0.68 / 0.61 / +10.9%
clickbench_q16/datafusion:vortex-file-compressed 155964709 / 153318888 / +1.7% 210049728 / 222694391 / -5.7% 0.74 / 0.69 / +7.8%
clickbench_q17/datafusion:vortex-file-compressed 151585998 / 153766540 / -1.4% 219331810 / 233396981 / -6.0% 0.69 / 0.66 / +4.9%
clickbench_q18/datafusion:vortex-file-compressed 299224636 / 295032098 / +1.4% 373914102 / 398720609 / -6.2% 0.80 / 0.74 / +8.1%
clickbench_q19/datafusion:vortex-file-compressed 8039247 / 8264734 / -2.7% 45878324 / 52445244 / -12.5% 🟢 0.18 / 0.16 / +11.2%
clickbench_q20/datafusion:vortex-file-compressed 70911641 / 69601585 / +1.9% 247365134 / 255050274 / -3.0% 0.29 / 0.27 / +5.0%
clickbench_q21/datafusion:vortex-file-compressed 79520020 / 79545494 / -0.0% 289305291 / 303920860 / -4.8% 0.27 / 0.26 / +5.0%
clickbench_q22/datafusion:vortex-file-compressed 91701118 / 92281360 / -0.6% 399876967 / 409072596 / -2.2% 0.23 / 0.23 / +1.7%
clickbench_q23/datafusion:vortex-file-compressed 191385114 / 186630780 / +2.5% 566149051 / 582812959 / -2.9% 0.34 / 0.32 / +5.6%
clickbench_q24/datafusion:vortex-file-compressed 15444250 / 15971598 / -3.3% 67702984 / 75248446 / -10.0% 🟢 0.23 / 0.21 / +7.5%
clickbench_q25/datafusion:vortex-file-compressed 19729978 / 19327944 / +2.1% 61023580 / 65544239 / -6.9% 0.32 / 0.29 / +9.6%
clickbench_q26/datafusion:vortex-file-compressed 17234608 / 16570822 / +4.0% 65130795 / 69223643 / -5.9% 0.26 / 0.24 / +10.5%
clickbench_q27/datafusion:vortex-file-compressed 101383166 / 104236139 / -2.7% 255123638 / 267629648 / -4.7% 0.40 / 0.39 / +2.0%
clickbench_q28/datafusion:vortex-file-compressed 553488758 / 554970976 / -0.3% 638204605 / 640981632 / -0.4% 0.87 / 0.87 / +0.2%
clickbench_q29/datafusion:vortex-file-compressed 23946782 / 24996223 / -4.2% 54067130 / 58416640 / -7.4% 0.44 / 0.43 / +3.5%
clickbench_q30/datafusion:vortex-file-compressed 57396974 / 58566897 / -2.0% 105716215 / 115415423 / -8.4% 0.54 / 0.51 / +7.0%
clickbench_q31/datafusion:vortex-file-compressed 65680519 / 66520857 / -1.3% 154763474 / 163467318 / -5.3% 0.42 / 0.41 / +4.3%
clickbench_q32/datafusion:vortex-file-compressed 288881277 / 288811144 / +0.0% 388017085 / 398601613 / -2.7% 0.74 / 0.72 / +2.8%
clickbench_q33/datafusion:vortex-file-compressed 319795602 / 323220982 / -1.1% 424202664 / 442745470 / -4.2% 0.75 / 0.73 / +3.3%
clickbench_q34/datafusion:vortex-file-compressed 324707788 / 323787036 / +0.3% 434010571 / 439727390 / -1.3% 0.75 / 0.74 / +1.6%
clickbench_q35/datafusion:vortex-file-compressed 57341812 / 56474642 / +1.5% 89226520 / 95239632 / -6.3% 0.64 / 0.59 / +8.4%
clickbench_q36/datafusion:vortex-file-compressed 37941902 / 37786497 / +0.4% 64813392 / 70220251 / -7.7% 0.59 / 0.54 / +8.8%
clickbench_q37/datafusion:vortex-file-compressed 21171256 / 21112899 / +0.3% 45637759 / 52236069 / -12.6% 🟢 0.46 / 0.40 / +14.8%
clickbench_q38/datafusion:vortex-file-compressed 12754017 / 13238210 / -3.7% 39554037 / 45728369 / -13.5% 🟢 0.32 / 0.29 / +11.4%
clickbench_q39/datafusion:vortex-file-compressed 79157998 / 79489746 / -0.4% 111081802 / 117000884 / -5.1% 0.71 / 0.68 / +4.9%
clickbench_q40/datafusion:vortex-file-compressed 10655434 / 10678944 / -0.2% 34590787 / 38994551 / -11.3% 🟢 0.31 / 0.27 / +12.5%
clickbench_q41/datafusion:vortex-file-compressed 10000790 / 10146938 / -1.4% 33731962 / 37697537 / -10.5% 🟢 0.30 / 0.27 / +10.1%
clickbench_q42/datafusion:vortex-file-compressed 10295524 / 10580150 / -2.7% 34223143 / 40151466 / -14.8% 🟢 0.30 / 0.26 / +14.2%
datafusion / parquet / ns (0.996x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/datafusion:parquet 1237938 / 1252390 / -1.2% 48162141 / 49858886 / -3.4% 0.03 / 0.03 / +2.3%
clickbench_q01/datafusion:parquet 16729284 / 15787223 / +6.0% 71967172 / 67905785 / +6.0% 0.23 / 0.23 / -0.0%
clickbench_q02/datafusion:parquet 18857051 / 19202549 / -1.8% 81289887 / 80765269 / +0.6% 0.23 / 0.24 / -2.4%
clickbench_q03/datafusion:parquet 17311226 / 16825988 / +2.9% 76062542 / 79107161 / -3.8% 0.23 / 0.21 / +7.0%
clickbench_q04/datafusion:parquet 65151663 / 64070752 / +1.7% 125549730 / 123746561 / +1.5% 0.52 / 0.52 / +0.2%
clickbench_q05/datafusion:parquet 88021996 / 89193812 / -1.3% 151912713 / 155866707 / -2.5% 0.58 / 0.57 / +1.3%
clickbench_q06/datafusion:parquet 1235956 / 1271545 / -2.8% 48040051 / 50022393 / -4.0% 0.03 / 0.03 / +1.2%
clickbench_q07/datafusion:parquet 17163906 / 16725616 / +2.6% 66052353 / 67183619 / -1.7% 0.26 / 0.25 / +4.4%
clickbench_q08/datafusion:parquet 84377779 / 86360220 / -2.3% 151906878 / 161677570 / -6.0% 0.56 / 0.53 / +4.0%
clickbench_q09/datafusion:parquet 184630502 / 189811070 / -2.7% 254869210 / 272037328 / -6.3% 0.72 / 0.70 / +3.8%
clickbench_q10/datafusion:parquet 36576450 / 35867330 / +2.0% 96811206 / 99994609 / -3.2% 0.38 / 0.36 / +5.3%
clickbench_q11/datafusion:parquet 42141217 / 41799902 / +0.8% 104043185 / 107840232 / -3.5% 0.41 / 0.39 / +4.5%
clickbench_q12/datafusion:parquet 95410852 / 95834502 / -0.4% 155747174 / 164548538 / -5.3% 0.61 / 0.58 / +5.2%
clickbench_q13/datafusion:parquet 145403506 / 145974546 / -0.4% 220820984 / 229047253 / -3.6% 0.66 / 0.64 / +3.3%
clickbench_q14/datafusion:parquet 101193027 / 102005408 / -0.8% 169512850 / 171671822 / -1.3% 0.60 / 0.59 / +0.5%
clickbench_q15/datafusion:parquet 71608165 / 71461450 / +0.2% 130175048 / 130790451 / -0.5% 0.55 / 0.55 / +0.7%
clickbench_q16/datafusion:parquet 154939452 / 156397034 / -0.9% 229976069 / 233902360 / -1.7% 0.67 / 0.67 / +0.8%
clickbench_q17/datafusion:parquet 155022700 / 157819059 / -1.8% 231488617 / 236051334 / -1.9% 0.67 / 0.67 / +0.2%
clickbench_q18/datafusion:parquet 307487234 / 307951774 / -0.2% 410434115 / 413174846 / -0.7% 0.75 / 0.75 / +0.5%
clickbench_q19/datafusion:parquet 18760652 / 19050488 / -1.5% 72784498 / 77455643 / -6.0% 0.26 / 0.25 / +4.8%
clickbench_q20/datafusion:parquet 177098547 / 179151449 / -1.1% 274015525 / 286433806 / -4.3% 0.65 / 0.63 / +3.3%
clickbench_q21/datafusion:parquet 172180028 / 172313051 / -0.1% 288174248 / 303072635 / -4.9% 0.60 / 0.57 / +5.1%
clickbench_q22/datafusion:parquet 258132790 / 263455818 / -2.0% 408788700 / 429668613 / -4.9% 0.63 / 0.61 / +3.0%
clickbench_q23/datafusion:parquet 1009501206 / 1015026952 / -0.5% 1177136370 / 1186808838 / -0.8% 0.86 / 0.86 / +0.3%
clickbench_q24/datafusion:parquet 30174817 / 30571946 / -1.3% 86091272 / 89809691 / -4.1% 0.35 / 0.34 / +3.0%
clickbench_q25/datafusion:parquet 37324408 / 37111422 / +0.6% 99250545 / 102893012 / -3.5% 0.38 / 0.36 / +4.3%
clickbench_q26/datafusion:parquet 30401663 / 31127172 / -2.3% 85243031 / 89186979 / -4.4% 0.36 / 0.35 / +2.2%
clickbench_q27/datafusion:parquet 192895383 / 193620522 / -0.4% 277620908 / 289723884 / -4.2% 0.69 / 0.67 / +4.0%
clickbench_q28/datafusion:parquet 619857272 / 617527624 / +0.4% 693876546 / 702331735 / -1.2% 0.89 / 0.88 / +1.6%
clickbench_q29/datafusion:parquet 28485739 / 28774798 / -1.0% 88725938 / 90223686 / -1.7% 0.32 / 0.32 / +0.7%
clickbench_q30/datafusion:parquet 89935306 / 89604524 / +0.4% 164522593 / 173512200 / -5.2% 0.55 / 0.52 / +5.9%
clickbench_q31/datafusion:parquet 102030556 / 103528343 / -1.4% 188028694 / 200819036 / -6.4% 0.54 / 0.52 / +5.3%
clickbench_q32/datafusion:parquet 303545305 / 308568780 / -1.6% 424110533 / 431256499 / -1.7% 0.72 / 0.72 / +0.0%
clickbench_q33/datafusion:parquet 375363888 / 377940900 / -0.7% 466315589 / 479131033 / -2.7% 0.80 / 0.79 / +2.0%
clickbench_q34/datafusion:parquet 376520527 / 379057750 / -0.7% 471140709 / 480750972 / -2.0% 0.80 / 0.79 / +1.4%
clickbench_q35/datafusion:parquet 65703426 / 65543578 / +0.2% 121809370 / 124134655 / -1.9% 0.54 / 0.53 / +2.2%
clickbench_q36/datafusion:parquet 73215784 / 73387590 / -0.2% 134779204 / 135774569 / -0.7% 0.54 / 0.54 / +0.5%
clickbench_q37/datafusion:parquet 35155010 / 35617999 / -1.3% 89801627 / 91810520 / -2.2% 0.39 / 0.39 / +0.9%
clickbench_q38/datafusion:parquet 47613380 / 47860311 / -0.5% 105251769 / 110026773 / -4.3% 0.45 / 0.43 / +4.0%
clickbench_q39/datafusion:parquet 143747196 / 143836521 / -0.1% 204150788 / 214263836 / -4.7% 0.70 / 0.67 / +4.9%
clickbench_q40/datafusion:parquet 19842294 / 20048918 / -1.0% 75451441 / 78849852 / -4.3% 0.26 / 0.25 / +3.4%
clickbench_q41/datafusion:parquet 18693045 / 18839682 / -0.8% 74953219 / 78602562 / -4.6% 0.25 / 0.24 / +4.1%
clickbench_q42/datafusion:parquet 19324382 / 19448928 / -0.6% 73504298 / 80271917 / -8.4% 0.26 / 0.24 / +8.5%
duckdb / vortex-file-compressed / ns (0.997x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/duckdb:vortex-file-compressed 2246636 / 2316356 / -3.0% 23641108 / 23309937 / +1.4% 0.10 / 0.10 / -4.4%
clickbench_q01/duckdb:vortex-file-compressed 7608596 / 7539062 / +0.9% 28088272 / 26697318 / +5.2% 0.27 / 0.28 / -4.1%
clickbench_q02/duckdb:vortex-file-compressed 9183714 / 8955982 / +2.5% 35599048 / 35695040 / -0.3% 0.26 / 0.25 / +2.8%
clickbench_q03/duckdb:vortex-file-compressed 14055928 / 13975386 / +0.6% 47146643 / 49335517 / -4.4% 0.30 / 0.28 / +5.2%
clickbench_q04/duckdb:vortex-file-compressed 56870457 / 58896211 / -3.4% 89565818 / 92282346 / -2.9% 0.63 / 0.64 / -0.5%
clickbench_q05/duckdb:vortex-file-compressed 66548707 / 67206239 / -1.0% 96549833 / 96372538 / +0.2% 0.69 / 0.70 / -1.2%
clickbench_q06/duckdb:vortex-file-compressed 2636254 / 2622598 / +0.5% 31702012 / 29238338 / +8.4% 0.08 / 0.09 / -7.3%
clickbench_q07/duckdb:vortex-file-compressed 9300782 / 9600068 / -3.1% 29545519 / 29770566 / -0.8% 0.31 / 0.32 / -2.4%
clickbench_q08/duckdb:vortex-file-compressed 69179025 / 70062960 / -1.3% 101797640 / 105800127 / -3.8% 0.68 / 0.66 / +2.6%
clickbench_q09/duckdb:vortex-file-compressed 92746171 / 94451218 / -1.8% 130142361 / 127163205 / +2.3% 0.71 / 0.74 / -4.1%
clickbench_q10/duckdb:vortex-file-compressed 28131792 / 27782772 / +1.3% 64828348 / 72279743 / -10.3% 🟢 0.43 / 0.38 / +12.9%
clickbench_q11/duckdb:vortex-file-compressed 30778926 / 30687561 / +0.3% 67268880 / 74359750 / -9.5% 0.46 / 0.41 / +10.9%
clickbench_q12/duckdb:vortex-file-compressed 71718374 / 72237397 / -0.7% 102972246 / 103652304 / -0.7% 0.70 / 0.70 / -0.1%
clickbench_q13/duckdb:vortex-file-compressed 125140960 / 127851810 / -2.1% 161892846 / 168822427 / -4.1% 0.77 / 0.76 / +2.1%
clickbench_q14/duckdb:vortex-file-compressed 76382316 / 76230723 / +0.2% 104451317 / 106914135 / -2.3% 0.73 / 0.71 / +2.6%
clickbench_q15/duckdb:vortex-file-compressed 76100357 / 76408312 / -0.4% 107681275 / 108616047 / -0.9% 0.71 / 0.70 / +0.5%
clickbench_q16/duckdb:vortex-file-compressed 140950308 / 140833212 / +0.1% 174577948 / 178949293 / -2.4% 0.81 / 0.79 / +2.6%
clickbench_q17/duckdb:vortex-file-compressed 141069799 / 145359782 / -3.0% 175088826 / 181211144 / -3.4% 0.81 / 0.80 / +0.4%
clickbench_q18/duckdb:vortex-file-compressed 257308932 / 256850284 / +0.2% 311800495 / 313893640 / -0.7% 0.83 / 0.82 / +0.9%
clickbench_q19/duckdb:vortex-file-compressed 9468834 / 9464116 / +0.0% 36675751 / 37478588 / -2.1% 0.26 / 0.25 / +2.2%
clickbench_q20/duckdb:vortex-file-compressed 68377837 / 69316572 / -1.4% 163391227 / 170472456 / -4.2% 0.42 / 0.41 / +2.9%
clickbench_q21/duckdb:vortex-file-compressed 84435002 / 87218998 / -3.2% 185329394 / 201513382 / -8.0% 0.46 / 0.43 / +5.3%
clickbench_q22/duckdb:vortex-file-compressed 174598020 / 174846654 / -0.1% 331883063 / 367208660 / -9.6% 0.53 / 0.48 / +10.5%
clickbench_q23/duckdb:vortex-file-compressed 40944288 / 41038768 / -0.2% 86386246 / 82961725 / +4.1% 0.47 / 0.49 / -4.2%
clickbench_q24/duckdb:vortex-file-compressed 14538760 / 15160866 / -4.1% 37085658 / 34968266 / +6.1% 0.39 / 0.43 / -9.6%
clickbench_q25/duckdb:vortex-file-compressed 19286890 / 19345222 / -0.3% 51524394 / 52265793 / -1.4% 0.37 / 0.37 / +1.1%
clickbench_q26/duckdb:vortex-file-compressed 11608645 / 11666164 / -0.5% 33491973 / 37230065 / -10.0% 🟢 0.35 / 0.31 / +10.6%
clickbench_q27/duckdb:vortex-file-compressed 53906572 / 54071963 / -0.3% 165316899 / 179248733 / -7.8% 0.33 / 0.30 / +8.1%
clickbench_q28/duckdb:vortex-file-compressed 518946884 / 521783236 / -0.5% 565054211 / 584919271 / -3.4% 0.92 / 0.89 / +3.0%
clickbench_q29/duckdb:vortex-file-compressed 17716504 / 18144333 / -2.4% 45916617 / 41948890 / +9.5% 0.39 / 0.43 / -10.8%
clickbench_q30/duckdb:vortex-file-compressed 61426654 / 61457455 / -0.1% 104721481 / 112065220 / -6.6% 0.59 / 0.55 / +7.0%
clickbench_q31/duckdb:vortex-file-compressed 110361410 / 112605193 / -2.0% 185709312 / 176967995 / +4.9% 0.59 / 0.64 / -6.6%
clickbench_q32/duckdb:vortex-file-compressed 385650798 / 376110058 / +2.5% 418613551 / 406967585 / +2.9% 0.92 / 0.92 / -0.3%
clickbench_q33/duckdb:vortex-file-compressed 349946660 / 341811647 / +2.4% 419204740 / 401498204 / +4.4% 0.83 / 0.85 / -1.9%
clickbench_q34/duckdb:vortex-file-compressed 360299106 / 359692298 / +0.2% 421334324 / 421374281 / -0.0% 0.86 / 0.85 / +0.2%
clickbench_q35/duckdb:vortex-file-compressed 59007020 / 59653987 / -1.1% 86722342 / 90891251 / -4.6% 0.68 / 0.66 / +3.7%
clickbench_q36/duckdb:vortex-file-compressed 11487672 / 11699479 / -1.8% 33228393 / 31365789 / +5.9% 0.35 / 0.37 / -7.3%
clickbench_q37/duckdb:vortex-file-compressed 8592844 / 8226245 / +4.5% 27348123 / 27060925 / +1.1% 0.31 / 0.30 / +3.4%
clickbench_q38/duckdb:vortex-file-compressed 8930644 / 8761657 / +1.9% 29032368 / 28743146 / +1.0% 0.31 / 0.30 / +0.9%
clickbench_q39/duckdb:vortex-file-compressed 19391234 / 18934137 / +2.4% 39281672 / 41063311 / -4.3% 0.49 / 0.46 / +7.1%
clickbench_q40/duckdb:vortex-file-compressed 7693176 / 7565950 / +1.7% 28053445 / 27136661 / +3.4% 0.27 / 0.28 / -1.6%
clickbench_q41/duckdb:vortex-file-compressed 8669756 / 8713054 / -0.5% 28513158 / 29103500 / -2.0% 0.30 / 0.30 / +1.6%
clickbench_q42/duckdb:vortex-file-compressed 7585390 / 7453789 / +1.8% 27434722 / 29052559 / -5.6% 0.28 / 0.26 / +7.8%
duckdb / parquet / ns (1.011x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/duckdb:parquet 11069670 / 10659744 / +3.8% 18123689 / 18978727 / -4.5% 0.61 / 0.56 / +8.7%
clickbench_q01/duckdb:parquet 17420745 / 16886617 / +3.2% 24375640 / 25296916 / -3.6% 0.71 / 0.67 / +7.1%
clickbench_q02/duckdb:parquet 24828358 / 24098232 / +3.0% 35516026 / 37071171 / -4.2% 0.70 / 0.65 / +7.5%
clickbench_q03/duckdb:parquet 25240006 / 24782620 / +1.8% 40120848 / 39824349 / +0.7% 0.63 / 0.62 / +1.1%
clickbench_q04/duckdb:parquet 64562244 / 63683324 / +1.4% 85923310 / 77506227 / +10.9% 🔴 0.75 / 0.82 / -8.6%
clickbench_q05/duckdb:parquet 86463217 / 85826466 / +0.7% 105524565 / 107119166 / -1.5% 0.82 / 0.80 / +2.3%
clickbench_q06/duckdb:parquet 22766170 / 22226085 / +2.4% 31429177 / 33590690 / -6.4% 0.72 / 0.66 / +9.5%
clickbench_q07/duckdb:parquet 19574566 / 19221367 / +1.8% 26600633 / 25706882 / +3.5% 0.74 / 0.75 / -1.6%
clickbench_q08/duckdb:parquet 77117734 / 77522808 / -0.5% 93053356 / 92848832 / +0.2% 0.83 / 0.83 / -0.7%
clickbench_q09/duckdb:parquet 104603526 / 104704682 / -0.1% 127895775 / 125363634 / +2.0% 0.82 / 0.84 / -2.1%
clickbench_q10/duckdb:parquet 42951052 / 42519106 / +1.0% 59958555 / 59927647 / +0.1% 0.72 / 0.71 / +1.0%
clickbench_q11/duckdb:parquet 49132584 / 47981428 / +2.4% 66338756 / 65672662 / +1.0% 0.74 / 0.73 / +1.4%
clickbench_q12/duckdb:parquet 94997784 / 95665316 / -0.7% 112818054 / 114826720 / -1.7% 0.84 / 0.83 / +1.1%
clickbench_q13/duckdb:parquet 155455652 / 153472152 / +1.3% 177107677 / 175809711 / +0.7% 0.88 / 0.87 / +0.6%
clickbench_q14/duckdb:parquet 104883654 / 102001694 / +2.8% 132484283 / 130684375 / +1.4% 0.79 / 0.78 / +1.4%
clickbench_q15/duckdb:parquet 89988836 / 90067308 / -0.1% 106515915 / 104927022 / +1.5% 0.84 / 0.86 / -1.6%
clickbench_q16/duckdb:parquet 168228172 / 176342836 / -4.6% 186744390 / 188023501 / -0.7% 0.90 / 0.94 / -3.9%
clickbench_q17/duckdb:parquet 169919810 / 177250039 / -4.1% 220328849 / 192559334 / +14.4% 🔴 0.77 / 0.92 / -16.2%
clickbench_q18/duckdb:parquet 305798594 / 308633344 / -0.9% 343581022 / 337205069 / +1.9% 0.89 / 0.92 / -2.8%
clickbench_q19/duckdb:parquet 22166072 / 22413691 / -1.1% 32573747 / 29637447 / +9.9% 0.68 / 0.76 / -10.0%
clickbench_q20/duckdb:parquet 187297544 / 186276237 / +0.5% 236808191 / 238066293 / -0.5% 0.79 / 0.78 / +1.1%
clickbench_q21/duckdb:parquet 229622350 / 227868066 / +0.8% 269131047 / 270418878 / -0.5% 0.85 / 0.84 / +1.3%
clickbench_q22/duckdb:parquet 330463202 / 309103402 / +6.9% 354211153 / 367922241 / -3.7% 0.93 / 0.84 / +11.0%
clickbench_q23/duckdb:parquet 171069302 / 170096610 / +0.6% 261267216 / 249917834 / +4.5% 0.65 / 0.68 / -3.8%
clickbench_q24/duckdb:parquet 33172754 / 32568394 / +1.9% 42427436 / 50429952 / -15.9% 🟢 0.78 / 0.65 / +21.1%
clickbench_q25/duckdb:parquet 45918882 / 45180834 / +1.6% 64111752 / 65172683 / -1.6% 0.72 / 0.69 / +3.3%
clickbench_q26/duckdb:parquet 32362501 / 33653694 / -3.8% 50912944 / 48840762 / +4.2% 0.64 / 0.69 / -7.8%
clickbench_q27/duckdb:parquet 201953038 / 194425344 / +3.9% 235485815 / 234323287 / +0.5% 0.86 / 0.83 / +3.4%
clickbench_q28/duckdb:parquet 999973306 / 993376018 / +0.7% 1010713239 / 978268921 / +3.3% 0.99 / 1.02 / -2.6%
clickbench_q29/duckdb:parquet 26472152 / 26075564 / +1.5% 35008718 / 35006863 / +0.0% 0.76 / 0.74 / +1.5%
clickbench_q30/duckdb:parquet 109748502 / 109086891 / +0.6% 143175543 / 141744231 / +1.0% 0.77 / 0.77 / -0.4%
clickbench_q31/duckdb:parquet 141033480 / 149473362 / -5.6% 189955179 / 181919472 / +4.4% 0.74 / 0.82 / -9.6%
clickbench_q32/duckdb:parquet 396832904 / 407535979 / -2.6% 439717966 / 415126757 / +5.9% 0.90 / 0.98 / -8.1%
clickbench_q33/duckdb:parquet 507939864 / 487845054 / +4.1% 489414131 / 477973659 / +2.4% 1.04 / 1.02 / +1.7%
clickbench_q34/duckdb:parquet 501071530 / 483134955 / +3.7% 495763298 / 474109910 / +4.6% 1.01 / 1.02 / -0.8%
clickbench_q35/duckdb:parquet 64907166 / 64853068 / +0.1% 79415741 / 78830710 / +0.7% 0.82 / 0.82 / -0.7%
clickbench_q36/duckdb:parquet 32613604 / 31689553 / +2.9% 39858052 / 40649108 / -1.9% 0.82 / 0.78 / +5.0%
clickbench_q37/duckdb:parquet 26430496 / 25055420 / +5.5% 34393783 / 33971986 / +1.2% 0.77 / 0.74 / +4.2%
clickbench_q38/duckdb:parquet 25724519 / 25376014 / +1.4% 33751785 / 34548555 / -2.3% 0.76 / 0.73 / +3.8%
clickbench_q39/duckdb:parquet 55624170 / 55054473 / +1.0% 63619829 / 65475996 / -2.8% 0.87 / 0.84 / +4.0%
clickbench_q40/duckdb:parquet 17014809 / 16160134 / +5.3% 25415493 / 25043946 / +1.5% 0.67 / 0.65 / +3.7%
clickbench_q41/duckdb:parquet 16379218 / 16865376 / -2.9% 24465437 / 24902932 / -1.8% 0.67 / 0.68 / -1.1%
clickbench_q42/duckdb:parquet 17145720 / 15796464 / +8.5% 25076774 / 26446483 / -5.2% 0.68 / 0.60 / +14.5%

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on S3 📖

Commits: PR 929c02f6 vs base 19dcd2f2
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -3.8%
Engines: DataFusion No clear signal (-1.9%, environment too noisy confidence) · DuckDB No clear signal (-5.6%, environment too noisy confidence)
Vortex (geomean): hot 1.012x ➖ · cold 1.046x ➖
Parquet (geomean): hot 1.051x ➖ · cold 1.025x ➖
Shifts: Parquet (control) +5.1% · Median polish +2.5%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.

datafusion / vortex-file-compressed / ns (1.004x ➖, 1↑ 1↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-file-compressed 212110890 / 225101151 / -5.8% 947222318 / 819874295 / +15.5% 0.22 / 0.27 / -18.4%
tpch_q02/datafusion:vortex-file-compressed 331893103 / 318534689 / +4.2% 582304723 / 723751029 / -19.5% 0.57 / 0.44 / +29.5%
tpch_q03/datafusion:vortex-file-compressed 387732494 / 377368788 / +2.7% 951691731 / 999232984 / -4.8% 0.41 / 0.38 / +7.9%
tpch_q04/datafusion:vortex-file-compressed 225264132 / 198635278 / +13.4% 540400146 / 477424781 / +13.2% 0.42 / 0.42 / +0.2%
tpch_q05/datafusion:vortex-file-compressed 377431663 / 393827139 / -4.2% 624977335 / 629883322 / -0.8% 0.60 / 0.63 / -3.4%
tpch_q06/datafusion:vortex-file-compressed 164564527 / 143980039 / +14.3% 366889280 / 326680898 / +12.3% 0.45 / 0.44 / +1.8%
tpch_q07/datafusion:vortex-file-compressed 429762270 / 459619010 / -6.5% 601703382 / 603682077 / -0.3% 0.71 / 0.76 / -6.2%
tpch_q08/datafusion:vortex-file-compressed 878993757 / 480151339 / +83.1% 🔴 1814729566 / 646130135 / +180.9% 🔴 0.48 / 0.74 / -34.8%
tpch_q09/datafusion:vortex-file-compressed 251358165 / 280196019 / -10.3% 482970758 / 569775944 / -15.2% 0.52 / 0.49 / +5.8%
tpch_q10/datafusion:vortex-file-compressed 418765506 / 335800983 / +24.7% 519285438 / 563854563 / -7.9% 0.81 / 0.60 / +35.4%
tpch_q11/datafusion:vortex-file-compressed 274362111 / 253028894 / +8.4% 468630954 / 411167746 / +14.0% 0.59 / 0.62 / -4.9%
tpch_q12/datafusion:vortex-file-compressed 267783321 / 247200676 / +8.3% 427692062 / 424206320 / +0.8% 0.63 / 0.58 / +7.4%
tpch_q13/datafusion:vortex-file-compressed 137888656 / 135457122 / +1.8% 367869066 / 327371805 / +12.4% 0.37 / 0.41 / -9.4%
tpch_q14/datafusion:vortex-file-compressed 186597129 / 168918289 / +10.5% 356146301 / 339885825 / +4.8% 0.52 / 0.50 / +5.4%
tpch_q15/datafusion:vortex-file-compressed 301461516 / 313396373 / -3.8% 604034980 / 671975736 / -10.1% 0.50 / 0.47 / +7.0%
tpch_q16/datafusion:vortex-file-compressed 142912083 / 154269699 / -7.4% 293962973 / 342273739 / -14.1% 0.49 / 0.45 / +7.9%
tpch_q17/datafusion:vortex-file-compressed 273169988 / 235285287 / +16.1% 486327373 / 448085308 / +8.5% 0.56 / 0.53 / +7.0%
tpch_q18/datafusion:vortex-file-compressed 577128407 / 577341508 / -0.0% 826146287 / 832055263 / -0.7% 0.70 / 0.69 / +0.7%
tpch_q19/datafusion:vortex-file-compressed 254904439 / 276418853 / -7.8% 486070478 / 444435959 / +9.4% 0.52 / 0.62 / -15.7%
tpch_q20/datafusion:vortex-file-compressed 383008054 / 306719206 / +24.9% 674296626 / 675470284 / -0.2% 0.57 / 0.45 / +25.1%
tpch_q21/datafusion:vortex-file-compressed 452283999 / 599463599 / -24.6% 641742076 / 958500706 / -33.0% 🟢 0.70 / 0.63 / +12.7%
tpch_q22/datafusion:vortex-file-compressed 159310649 / 415277462 / -61.6% 🟢 394717213 / 1050283869 / -62.4% 🟢 0.40 / 0.40 / +2.1%
datafusion / parquet / ns (1.023x ➖, 1↑ 1↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:parquet 165700140 / 241682856 / -31.4% 🟢 940277867 / 487531007 / +92.9% 🔴 0.18 / 0.50 / -64.5%
tpch_q02/datafusion:parquet 323689379 / 269956217 / +19.9% 555639685 / 609423406 / -8.8% 0.58 / 0.44 / +31.5%
tpch_q03/datafusion:parquet 256367757 / 251068913 / +2.1% 663663951 / 663935581 / -0.0% 0.39 / 0.38 / +2.2%
tpch_q04/datafusion:parquet 142909501 / 144199922 / -0.9% 307912274 / 450930959 / -31.7% 🟢 0.46 / 0.32 / +45.1%
tpch_q05/datafusion:parquet 343086436 / 355148108 / -3.4% 465753397 / 521470490 / -10.7% 0.74 / 0.68 / +8.2%
tpch_q06/datafusion:parquet 121526388 / 123561347 / -1.6% 236895462 / 220313766 / +7.5% 0.51 / 0.56 / -8.5%
tpch_q07/datafusion:parquet 345282929 / 374411551 / -7.8% 499002976 / 517618666 / -3.6% 0.69 / 0.72 / -4.3%
tpch_q08/datafusion:parquet 450271731 / 425276503 / +5.9% 665837626 / 533330822 / +24.8% 0.68 / 0.80 / -15.2%
tpch_q09/datafusion:parquet 407380562 / 396498657 / +2.7% 553573510 / 623882092 / -11.3% 0.74 / 0.64 / +15.8%
tpch_q10/datafusion:parquet 334896212 / 311949145 / +7.4% 595331497 / 452955570 / +31.4% 🔴 0.56 / 0.69 / -18.3%
tpch_q11/datafusion:parquet 275146251 / 247927590 / +11.0% 384413749 / 351353202 / +9.4% 0.72 / 0.71 / +1.4%
tpch_q12/datafusion:parquet 165089248 / 165576091 / -0.3% 332115783 / 290563979 / +14.3% 0.50 / 0.57 / -12.8%
tpch_q13/datafusion:parquet 277212125 / 278650002 / -0.5% 479898778 / 475941381 / +0.8% 0.58 / 0.59 / -1.3%
tpch_q14/datafusion:parquet 184762101 / 215575299 / -14.3% 324712228 / 465756460 / -30.3% 🟢 0.57 / 0.46 / +22.9%
tpch_q15/datafusion:parquet 355912683 / 314097726 / +13.3% 617240936 / 437056563 / +41.2% 🔴 0.58 / 0.72 / -19.8%
tpch_q16/datafusion:parquet 101572155 / 102227252 / -0.6% 212750361 / 196549578 / +8.2% 0.48 / 0.52 / -8.2%
tpch_q17/datafusion:parquet 280806611 / 295249476 / -4.9% 470908997 / 424664867 / +10.9% 0.60 / 0.70 / -14.2%
tpch_q18/datafusion:parquet 326389728 / 315661264 / +3.4% 475248102 / 534625462 / -11.1% 0.69 / 0.59 / +16.3%
tpch_q19/datafusion:parquet 174992003 / 160275591 / +9.2% 294691513 / 288539952 / +2.1% 0.59 / 0.56 / +6.9%
tpch_q20/datafusion:parquet 290681938 / 259105255 / +12.2% 397770652 / 375574076 / +5.9% 0.73 / 0.69 / +5.9%
tpch_q21/datafusion:parquet 358832248 / 328233636 / +9.3% 505244796 / 595891416 / -15.2% 0.71 / 0.55 / +28.9%
tpch_q22/datafusion:parquet 210343074 / 151945497 / +38.4% 🔴 349073425 / 336477683 / +3.7% 0.60 / 0.45 / +33.4%
duckdb / vortex-file-compressed / ns (1.019x ➖, 1↑ 1↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-file-compressed 251673730 / 242713738 / +3.7% 300008402 / 290906301 / +3.1% 0.84 / 0.83 / +0.5%
tpch_q02/duckdb:vortex-file-compressed 450804077 / 395133430 / +14.1% 527900340 / 503306863 / +4.9% 0.85 / 0.79 / +8.8%
tpch_q03/duckdb:vortex-file-compressed 416463087 / 403958513 / +3.1% 451161062 / 506631264 / -10.9% 0.92 / 0.80 / +15.8%
tpch_q04/duckdb:vortex-file-compressed 289258958 / 275371920 / +5.0% 308936932 / 264656780 / +16.7% 0.94 / 1.04 / -10.0%
tpch_q05/duckdb:vortex-file-compressed 630423256 / 554927492 / +13.6% 665421294 / 635027040 / +4.8% 0.95 / 0.87 / +8.4%
tpch_q06/duckdb:vortex-file-compressed 177084561 / 264305141 / -33.0% 🟢 276691511 / 217836159 / +27.0% 0.64 / 1.21 / -47.3%
tpch_q07/duckdb:vortex-file-compressed 489385510 / 470551583 / +4.0% 649601331 / 591002499 / +9.9% 0.75 / 0.80 / -5.4%
tpch_q08/duckdb:vortex-file-compressed 576882536 / 677708070 / -14.9% 738619103 / 790424730 / -6.6% 0.78 / 0.86 / -8.9%
tpch_q09/duckdb:vortex-file-compressed 501464337 / 414257896 / +21.1% 485266880 / 421773998 / +15.1% 1.03 / 0.98 / +5.2%
tpch_q10/duckdb:vortex-file-compressed 545486270 / 565809056 / -3.6% 832193790 / 532322473 / +56.3% 🔴 0.66 / 1.06 / -38.3%
tpch_q11/duckdb:vortex-file-compressed 311799426 / 331444316 / -5.9% 345118092 / 326000481 / +5.9% 0.90 / 1.02 / -11.1%
tpch_q12/duckdb:vortex-file-compressed 593816762 / 300240325 / +97.8% 🔴 344406320 / 293824998 / +17.2% 1.72 / 1.02 / +68.7%
tpch_q13/duckdb:vortex-file-compressed 228147268 / 234634397 / -2.8% 257539413 / 285599348 / -9.8% 0.89 / 0.82 / +7.8%
tpch_q14/duckdb:vortex-file-compressed 239510616 / 226387254 / +5.8% 299168685 / 250340989 / +19.5% 0.80 / 0.90 / -11.5%
tpch_q15/duckdb:vortex-file-compressed 251579805 / 244667904 / +2.8% 299284091 / 274125844 / +9.2% 0.84 / 0.89 / -5.8%
tpch_q16/duckdb:vortex-file-compressed 208779959 / 207480636 / +0.6% 291830752 / 212283893 / +37.5% 🔴 0.72 / 0.98 / -26.8%
tpch_q17/duckdb:vortex-file-compressed 335004922 / 304429247 / +10.0% 487765889 / 400945885 / +21.7% 0.69 / 0.76 / -9.5%
tpch_q18/duckdb:vortex-file-compressed 362037817 / 442479898 / -18.2% 483619615 / 396598793 / +21.9% 0.75 / 1.12 / -32.9%
tpch_q19/duckdb:vortex-file-compressed 284797460 / 347257411 / -18.0% 365653440 / 333079622 / +9.8% 0.78 / 1.04 / -25.3%
tpch_q20/duckdb:vortex-file-compressed 353363339 / 356522154 / -0.9% 466264947 / 525593522 / -11.3% 0.76 / 0.68 / +11.7%
tpch_q21/duckdb:vortex-file-compressed 673825699 / 669563383 / +0.6% 997277384 / 768477162 / +29.8% 0.68 / 0.87 / -22.5%
tpch_q22/duckdb:vortex-file-compressed 297858984 / 288118977 / +3.4% 302252702 / 321455013 / -6.0% 0.99 / 0.90 / +9.9%
duckdb / parquet / ns (1.080x ➖, 0↑ 4↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:parquet 232498392 / 157928349 / +47.2% 🔴 286848047 / 272690714 / +5.2% 0.81 / 0.58 / +40.0%
tpch_q02/duckdb:parquet 512637491 / 466918592 / +9.8% 541747519 / 471472543 / +14.9% 0.95 / 0.99 / -4.5%
tpch_q03/duckdb:parquet 438465517 / 467561365 / -6.2% 478968513 / 447868293 / +6.9% 0.92 / 1.04 / -12.3%
tpch_q04/duckdb:parquet 275820894 / 241132863 / +14.4% 431357208 / 346699496 / +24.4% 0.64 / 0.70 / -8.1%
tpch_q05/duckdb:parquet 632405372 / 601681209 / +5.1% 638263686 / 642800358 / -0.7% 0.99 / 0.94 / +5.9%
tpch_q06/duckdb:parquet 175288383 / 157205162 / +11.5% 252839723 / 194250011 / +30.2% 🔴 0.69 / 0.81 / -14.3%
tpch_q07/duckdb:parquet 476716925 / 483254002 / -1.4% 680600255 / 1107993230 / -38.6% 🟢 0.70 / 0.44 / +60.6%
tpch_q08/duckdb:parquet 678634134 / 654285856 / +3.7% 731311742 / 669007016 / +9.3% 0.93 / 0.98 / -5.1%
tpch_q09/duckdb:parquet 601131589 / 644494164 / -6.7% 639136844 / 718771676 / -11.1% 0.94 / 0.90 / +4.9%
tpch_q10/duckdb:parquet 569334374 / 530902132 / +7.2% 627418230 / 637249426 / -1.5% 0.91 / 0.83 / +8.9%
tpch_q11/duckdb:parquet 257738576 / 303673870 / -15.1% 280969963 / 325318246 / -13.6% 0.92 / 0.93 / -1.7%
tpch_q12/duckdb:parquet 308923916 / 215585071 / +43.3% 🔴 358023992 / 266896912 / +34.1% 🔴 0.86 / 0.81 / +6.8%
tpch_q13/duckdb:parquet 483290782 / 358775974 / +34.7% 🔴 533841359 / 449605311 / +18.7% 0.91 / 0.80 / +13.4%
tpch_q14/duckdb:parquet 272948438 / 328300491 / -16.9% 292913861 / 540192412 / -45.8% 🟢 0.93 / 0.61 / +53.3%
tpch_q15/duckdb:parquet 240926574 / 193853728 / +24.3% 329312250 / 275208554 / +19.7% 0.73 / 0.70 / +3.9%
tpch_q16/duckdb:parquet 294607498 / 265133880 / +11.1% 300729708 / 323161613 / -6.9% 0.98 / 0.82 / +19.4%
tpch_q17/duckdb:parquet 331501905 / 350853382 / -5.5% 385904488 / 381556280 / +1.1% 0.86 / 0.92 / -6.6%
tpch_q18/duckdb:parquet 384809383 / 345691529 / +11.3% 409511894 / 368650896 / +11.1% 0.94 / 0.94 / +0.2%
tpch_q19/duckdb:parquet 444994901 / 299979187 / +48.3% 🔴 629064404 / 345698365 / +82.0% 🔴 0.71 / 0.87 / -18.5%
tpch_q20/duckdb:parquet 471581368 / 506822076 / -7.0% 552711812 / 594115221 / -7.0% 0.85 / 0.85 / +0.0%
tpch_q21/duckdb:parquet 463998068 / 478529087 / -3.0% 512097917 / 523115452 / -2.1% 0.91 / 0.91 / -1.0%
tpch_q22/duckdb:parquet 345535466 / 351512104 / -1.7% 330402577 / 431678340 / -23.5% 1.05 / 0.81 / +28.4%

@joseph-isaacs
joseph-isaacs enabled auto-merge (squash) October 30, 2026 18:46
@joseph-isaacs
joseph-isaacs merged commit 493967c into develop Oct 30, 2026
166 of 176 checks passed
@joseph-isaacs
joseph-isaacs deleted the ji/elegant-bardeen-y7z1cu branch October 30, 2026 18:48
robert3005 pushed a commit that referenced this pull request Nov 2, 2026
Follows #11055 (FineWeb). This moves every remaining benchmark except
StatPopGen and PolarSignals onto the planner-driven V2 scan. After this
PR, those two are the only benchmarks still on V1.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants