Repository navigation
Conversation
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
commented
Nov 5, 2026
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (0.978x ➖, 2↑ 1↓)
|
commented
Nov 5, 2026
Benchmarks: Clickbench Sorted on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.989x ➖, 1↑ 0↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.989x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.975x ➖, 1↑ 0↓)
duckdb / duckdb / ns (1.018x ➖, 0↑ 1↓)
|
commented
Nov 5, 2026
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
datafusion / vortex-compact / ns (0.992x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.997x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.999x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.997x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.999x ➖, 0↑ 0↓)
duckdb / duckdb / ns (0.999x ➖, 0↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: Statistical and Population Genetics 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.847x ✅, 5↑ 0↓)
datafusion / vortex-compact / ns (0.869x ✅, 5↑ 0↓)
datafusion / parquet / ns (0.992x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.998x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.992x ➖, 2↑ 0↓)
duckdb / parquet / ns (1.003x ➖, 0↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: FineWeb S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.065x ➖, 0↑ 2↓)
datafusion / vortex-compact / ns (0.898x ➖, 1↑ 0↓)
datafusion / parquet / ns (0.871x ➖, 2↑ 1↓)
duckdb / vortex-file-compressed / ns (0.900x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.925x ➖, 1↑ 0↓)
duckdb / parquet / ns (0.968x ➖, 1↑ 2↓)
|
commented
Nov 5, 2026
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.003x ➖, 0↑ 0↓)
datafusion / vortex-compact / ns (0.998x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.997x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.999x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.999x ➖, 0↑ 0↓)
duckdb / duckdb / ns (1.002x ➖, 0↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: Appian on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.997x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.993x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.005x ➖, 0↑ 0↓)
duckdb / duckdb / ns (1.003x ➖, 0↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: JOB on NVME 📖Commits: PR How to read Verdict and Engines
duckdb / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.004x ➖, 1↑ 2↓)
duckdb / duckdb / ns (1.008x ➖, 0↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: TPC-DS SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.889x ✅, 39↑ 5↓)
datafusion / parquet / ns (1.001x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (1.001x ➖, 0↑ 1↓)
duckdb / parquet / ns (1.002x ➖, 0↑ 1↓)
|
commented
Nov 5, 2026
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.005x ➖, 2↑ 3↓)
datafusion / vortex-compact / ns (1.004x ➖, 0↑ 2↓)
datafusion / parquet / ns (1.002x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.014x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.011x ➖, 1↑ 1↓)
duckdb / parquet / ns (1.014x ➖, 0↑ 1↓)
duckdb / duckdb / ns (1.010x ➖, 0↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.003x ➖, 1↑ 1↓)
datafusion / parquet / ns (1.005x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.014x ➖, 0↑ 1↓)
duckdb / parquet / ns (1.007x ➖, 0↑ 1↓)
duckdb / duckdb / ns (1.004x ➖, 1↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.052x ➖, 0↑ 1↓)
datafusion / vortex-compact / ns (1.060x ➖, 0↑ 1↓)
datafusion / parquet / ns (0.995x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.024x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.001x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.996x ➖, 0↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.783x ➖, 5↑ 0↓)
datafusion / vortex-compact / ns (0.866x ➖, 4↑ 2↓)
datafusion / parquet / ns (0.988x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (0.857x ➖, 3↑ 0↓)
duckdb / vortex-compact / ns (1.010x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.845x ➖, 3↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: Vortex queries 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.997x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.010x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.994x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.985x ➖, 0↑ 0↓)
|
commented
Nov 5, 2026
Benchmarks: TPC-H SF=10 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.796x ➖, 6↑ 0↓)
datafusion / vortex-compact / ns (0.921x ➖, 3↑ 2↓)
datafusion / parquet / ns (0.860x ➖, 4↑ 1↓)
duckdb / vortex-file-compressed / ns (0.948x ➖, 2↑ 0↓)
duckdb / vortex-compact / ns (0.981x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.928x ➖, 2↑ 0↓)
|
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
0253169 to
e909ee5
Compare
Summary
Several distinct
list_transformlambdas over one list, or a lambda beside the list read whole, go into one elements read. The elements reader packs one value per lambda, plus the raw element when needed, and the lists come back as a struct of lists sharing the offsets and validity. Over the whole stack, the microbench geomean of 112 cases is 0.65x of develop.Changes
split_list_transform(list/expr.rs): packs the bodies as"0","1", ... and rewrites each use to its list.lists_over(list/reader.rs): one list per pack field, extracted lazily from the packed elements.Testing
list_transform_reads_only_what_its_body_uses::two_lambdasnow reads onlyaandf.random_list_uses_match_memorycovers two lambdas besidelist_sum,list_length,is_null, and the whole list.Follow-up
list_transformforis_nullandlist_length.