Skip to content

bench: one File Sizes group ranks every format against zstd Parquet - #227

Draft
lwwmanning wants to merge 6 commits into
mainfrom
wm/consolidate-file-size-metrics-0b3056
Draft

lwwmanning wants to merge 6 commits into
mainfrom
wm/consolidate-file-size-metrics-0b3056

Conversation

@lwwmanning

@lwwmanning lwwmanning commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

What changes

bench and the vortex.dev homepage. The Compression Size group is now File Sizes. Its summary ranks every format against zstd Parquet, on the compress-bench datasets and on TPC-DS at scale factor 1 (SF1), with the newest result of each format. The homepage size card reads it. Old Compression Size links still work. SQL File Sizes stays, and charts every SQL scale factor against Vortex.

Query summaries now rank a series that misses a query below every series that ran all of them. Lance skips TPC-DS q72, and the penalty alone let a skipped query cost too little. Snappy Parquet lines are now dotted, so they differ from DuckDB's dashed lines.

The methodology page now explains the two Parquet variants. Parquet is zstd level 3, and Parquet (Snappy) is Parquet as DuckDB writes it. ClickBench and FineWeb use their published files as Parquet.

The Snappy Parquet and compact Vortex sizes, and TPC-DS SF1's Snappy, Lance, and Arrow IPC results, come from vortex-data/vortex#11720.

How it was checked

bun run check:ci, tsc, the site and bench tests, the build, and knip pass. No screenshots yet.

  • Every commit is signed off (git commit -s), for the DCO check.
  • bun run check:ci passes.
  • Each new image or other asset has an annotation in REUSE.toml.

🤖 Generated with Claude Code

The Compression Size and SQL File Size groups showed the same metric,
file size per format, from two suites. One File Sizes group now charts
the compress-bench datasets and three SQL scopes: ClickBench, TPC-H at
SF10, and TPC-DS at SF1. Old Compression Size links redirect to it, and
the homepage card reads the group by its new or old name.

The summary compares each format with zstd Parquet on the compress-bench
datasets and on TPC-DS. Each format uses its newest commit that has zstd
Parquet. The wide-table and sparse-wide families count as one dataset
each. A format measured on only some datasets says how many, has no
total, and ranks after the complete formats. Parquet codec variants,
such as parquet-snappy, draw in Parquet's color with a dashed line.

Signed-off-by: Will Manning <will@willmanning.io>
File Sizes moves to the Compression Size group key, so old Compression
Size links open it. SQL File Sizes comes back on its own key: it charts
every SQL benchmark and scale factor and ranks formats against Vortex,
as before. Each group's info tooltip says how the two differ.

Signed-off-by: Will Manning <will@willmanning.io>
Signed-off-by: Will Manning <will@willmanning.io>
A series with no time for a query now ranks below every series that ran
all of them. The missing-series penalty alone scores a skipped query at
2x the best time, so a series could rank first by skipping the slowest
query. Lance skips TPC-DS q72, which it cannot run.

Snappy Parquet joins TPC-DS's queries, so a codec variant is now dotted,
and dash-dotted on DuckDB, to keep it apart from DuckDB's dashed Parquet.

Signed-off-by: Will Manning <will@willmanning.io>
@vercel

vercel Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
vortex.dev Ready Ready Preview Nov 21, 2026 2:34am UTC

Request Review

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

🔦 Lighthouse audit

URL Perf A11y Best practices SEO
/ 🟢 100 🟢 100 🟢 100 🟢 100
/bench 🟢 100 🟢 100 🟢 100 🟢 100
/bench/chart/ct.eyJrIjoiQ29tcHJlc3Npb25UaW1lIiwiZGF0YXNldCI6ImFyYWRlIiwiZGF0YXNldF92YXJpYW50IjpudWxsfQ 🟢 100 🟢 100 🟢 100 🟢 100
/blog 🟢 99 🟢 100 🟢 100 🟢 100
/blog/btrblocks-compressor 🟢 99 🟢 100 🟢 100 🟢 100

Run 44835922876 · 2026-11-20 23:22 UTC

Vercel adds its Toolbar to preview deployments, and the proxy's
x-vercel-skip-toolbar header does not stop it. The Toolbar loads from
vercel.live and Pusher, and its third-party cookies and issues cut Best
Practices to 0.78 on every audited page. The site loads neither host,
so Lighthouse now blocks both.

Signed-off-by: Will Manning <will@willmanning.io>
Parquet is zstd level 3, the size and throughput baseline. Parquet
(Snappy) is Parquet as DuckDB writes it by default, which TPC-DS SF1
queries and the compression benchmark sizes. ClickBench and FineWeb query
their published files as Parquet, and ClickBench's are Snappy. The page
also says that a query summary ranks a series that missed a query below
the complete ones, as Lance on TPC-DS q72.

The contract now names file sizes by format directory, not by codec.

Signed-off-by: Will Manning <will@willmanning.io>

This branch was successfully deployed

1 active deployment
Preview — 27097f82 Deployed Nov 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant