Repository navigation
fix: ignore zero-byte benchmark file identities - #9197
Conversation
Merging this PR will improve performance by 18.02%
Performance Changes
Tip Curious why this is faster? Comment Comparing Footnotes
|
ec1a6d0 to
3020476
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
3020476 to
6380aa5
Compare
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals Profiling 📖Vortex (geomean): 1.007x ➖ datafusion / vortex-file-compressed / ns (1.007x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.005x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.977x ➖, 4↑ 1↓)
duckdb / vortex-file-compressed / ns (0.999x ➖, 1↑ 0↓)
duckdb / parquet / ns (0.998x ➖, 0↑ 0↓)
File Size Changes (9 files changed, -43.9% overall, 0↑ 9↓)
Totals:
|
Benchmarks: TPC-H SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.002x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.996x ➖, 0↑ 1↓)
duckdb / vortex-compact / ns (0.978x ➖, 1↑ 0↓)
duckdb / parquet / ns (0.997x ➖, 0↑ 0↓)
File Size Changes (9 files changed, -56.1% overall, 0↑ 9↓)
Totals:
|
Benchmarks: Statistical and Population Genetics 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
duckdb / vortex-file-compressed / ns (0.984x ➖, 2↑ 1↓)
duckdb / parquet / ns (0.999x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -32.3% overall, 0↑ 2↓)
Totals:
|
Benchmarks: FineWeb NVMe 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (0.993x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.990x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.003x ➖, 1↑ 0↓)
duckdb / parquet / ns (0.992x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -53.7% overall, 0↑ 2↓)
Totals:
|
Benchmarks: Clickbench on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.004x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.006x ➖, 0↑ 1↓)
duckdb / vortex-compact / ns (1.002x ➖, 3↑ 3↓)
duckdb / parquet / ns (1.000x ➖, 0↑ 0↓)
File Size Changes (101 files changed, -60.8% overall, 0↑ 101↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.010x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.998x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.991x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.020x ➖, 0↑ 1↓)
File Size Changes (9 files changed, -56.0% overall, 0↑ 9↓)
Totals:
|
Benchmarks: Clickbench on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.016x ➖, 0↑ 2↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (1.005x ➖, 1↑ 3↓)
duckdb / parquet / ns (1.000x ➖, 0↑ 1↓)
File Size Changes (101 files changed, -39.2% overall, 0↑ 101↓)
Totals:
|
Benchmarks: FineWeb S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.125x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.316x ❌, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (0.955x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.005x ➖, 0↑ 0↓)
|
Benchmarks: TPC-DS SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (0.998x ➖, 1↑ 0↓)
datafusion / parquet / ns (1.015x ➖, 0↑ 6↓)
duckdb / vortex-compact / ns (1.013x ➖, 1↑ 3↓)
duckdb / parquet / ns (1.001x ➖, 3↑ 3↓)
File Size Changes (25 files changed, -56.5% overall, 0↑ 25↓)
Totals:
|
Benchmarks: Appian on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.001x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.993x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.994x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.003x ➖, 0↑ 0↓)
File Size Changes (10 files changed, -63.8% overall, 0↑ 10↓)
Totals:
|
Benchmarks: TPC-DS SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 3↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.048x ➖, 6↑ 22↓)
duckdb / parquet / ns (1.008x ➖, 2↑ 5↓)
File Size Changes (25 files changed, -43.5% overall, 0↑ 25↓)
Totals:
|
Benchmarks: FineWeb S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (0.992x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.031x ➖, 0↑ 1↓)
duckdb / vortex-compact / ns (1.006x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.647x ✅, 2↑ 0↓)
|
Benchmarks: FineWeb NVMe 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.979x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.902x ➖, 4↑ 0↓)
duckdb / parquet / ns (1.008x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -46.3% overall, 0↑ 2↓)
Totals:
|
Benchmarks: Clickbench Sorted on NVME 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.959x ➖, 2↑ 1↓)
datafusion / parquet / ns (0.993x ➖, 1↑ 2↓)
duckdb / vortex-file-compressed / ns (1.016x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.991x ➖, 0↑ 0↓)
File Size Changes (201 files changed, -42.8% overall, 42↑ 159↓)
Totals:
|
Benchmarks: TPC-H SF=10 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (0.946x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.936x ➖, 2↑ 1↓)
duckdb / vortex-compact / ns (0.960x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.936x ➖, 1↑ 0↓)
|
Benchmarks: TPC-H SF=1 on S3 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (0.983x ➖, 0↑ 1↓)
datafusion / parquet / ns (0.987x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.022x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.997x ➖, 0↑ 0↓)
|
Benchmarks: Clickbench Sorted on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (0.987x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.977x ➖, 1↑ 0↓)
duckdb / vortex-compact / ns (0.970x ➖, 1↑ 0↓)
duckdb / parquet / ns (0.974x ➖, 0↑ 0↓)
File Size Changes (201 files changed, -57.2% overall, 48↑ 153↓)
Totals:
|
Benchmarks: Statistical and Population Genetics 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
duckdb / vortex-compact / ns (0.996x ➖, 1↑ 2↓)
duckdb / parquet / ns (0.997x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -67.7% overall, 0↑ 2↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.091x ➖, 0↑ 3↓)
datafusion / parquet / ns (1.028x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (0.981x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.021x ➖, 0↑ 0↓)
|
Benchmarks: TPC-H SF=10 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.003x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.995x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.001x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.003x ➖, 1↑ 0↓)
File Size Changes (9 files changed, -44.0% overall, 0↑ 9↓)
Totals:
|
Fixes file-size reporting when either the baseline or HEAD records an empty file. The report previously dropped the zero-byte row and then treated its missing counterpart as
0 B, which could report a false deletion or a newly created file.File identities with a zero-byte record on either side are now excluded from both sides before totals and missing-file comparisons. The regression test covers both asymmetric cases.
Checks:
uv run --no-project --with orjson --with pandas pytest scripts/tests/test_benchmark_reporting.pyandruff check scripts/compare-benchmark-jsons.py scripts/tests/test_benchmark_reporting.py.