Repository navigation
Find libduckdb for the SQL benchmarks wherever cargo puts it - #11462
Conversation
Every SQL benchmark that runs DuckDB fails with "libduckdb.so: cannot open shared object file". Rust 1.100's cargo moved build script output from build/<pkg>-<hash>/out to build/<pkg>/<hash>/out. vortex-duckdb keeps its DuckDB cache two levels above OUT_DIR, so the library is now under build/vortex-duckdb/vortex-duckdb-cache/, and the artifact path build/vortex-duckdb-cache/... matches nothing. upload-artifact leaves an unmatched path out silently, so the bench jobs get duckdb-bench without the library that its absolute rpath points to. Match the library at any depth under target/release_debug/build, and fail the build job if it is missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniBAMeLpyH1yASJhrK1N Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
commented
Nov 13, 2026
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (1.006x ➖, 0↑ 0↓)
No file size changes detected. |
commented
Nov 13, 2026
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.998x ➖, 1↑ 1↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.006x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.004x ➖, 0↑ 0↓)
No file size changes detected. |
commented
Nov 13, 2026
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.009x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.002x ➖, 0↑ 0↓)
No file size changes detected. |
commented
Nov 13, 2026
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.988x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.001x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.994x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.994x ➖, 0↑ 0↓)
No file size changes detected. |
commented
Nov 13, 2026
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.006x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.003x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.013x ➖, 1↑ 0↓)
duckdb / parquet / ns (1.010x ➖, 1↑ 1↓)
No file size changes detected. |
commented
Nov 13, 2026
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.982x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.996x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.001x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.996x ➖, 0↑ 0↓)
No file size changes detected. |
commented
Nov 13, 2026
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.192x ➖, 0↑ 5↓)
datafusion / parquet / ns (1.028x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.077x ➖, 1↑ 3↓)
duckdb / parquet / ns (1.049x ➖, 0↑ 0↓)
|
commented
Nov 13, 2026
Benchmarks: TPC-DS SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.002x ➖, 1↑ 0↓)
datafusion / parquet / ns (1.003x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.995x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.999x ➖, 0↑ 1↓)
No baseline file sizes found for base commit. |
Summary
Every SQL benchmark that runs DuckDB fails with:
This started with the Rust 1.100 upgrade (#11446). Rust 1.100's cargo moved build script output from
build/<pkg>-<hash>/outtobuild/<pkg>/<hash>/out.vortex-duckdb/build.rskeeps its DuckDB cache two levels aboveOUT_DIR, so the library moved:target/release_debug/build/vortex-duckdb-cache/duckdb-lib-*/libduckdb.sotarget/release_debug/build/vortex-duckdb/vortex-duckdb-cache/duckdb-lib-*/libduckdb.soThe artifact path still names the old location. upload-artifact leaves out an unmatched path without an error, so the bench jobs get
duckdb-benchwithout the library that its absolute rpath points to.Changes
sql-bench-matrix.yml, build job:target/release_debug/build, so it works with both layouts. It also matches a DuckDB built from source as well as a prebuilt one. upload-artifact still roots the artifact attarget/release_debug, so the library unpacks at the path the rpath expects.libduckdb.sois missing. A missing library now stops the build, instead of letting every DuckDB benchmark fail later.A sturdier fix would be for
vortex-duckdb/build.rsto stop deriving its cache location fromOUT_DIR, since cargo can change that layout again. I kept this PR to the workflow.Checks:
yamllint --strict -c .yamllint.yamlpasses.OUT_DIR=.../target/debug/build/<pkg>/<hash>/out.Generated by Claude Code