Repository navigation
perf(scan): restrict split-boundary searches to the active row range - #10552
joseph-isaacs wants to merge 1 commit into
Conversation
`cut_at` scanned every chunk boundary on each call, even for a small subrange. Binary-search the first boundary after `rows.start` and stop at `rows.end`, so the search is O(log N + K) rather than O(N). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hai26WGsNERxRHXpA3JboE Signed-off-by: Claude <noreply@anthropic.com>
Merging this PR will improve performance by 34.84%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ |
Simulation | density_sweep_dense_runs[0.001] |
46.8 µs | 29 µs | +61.28% |
| ⚡ | WallTime | bitpack_blocked_compress_avx2 |
7.6 µs | 6.7 µs | +12.73% |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ji/wonderful-cannon-9rhahl (2d9daf6) with ji/scan-v2-4-wiring (9f5906e)
Footnotes
-
12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Codecov Report✅ All modified and coverable lines are covered by tests. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
|
Closing as superseded: Generated by Claude Code |
Summary
5/5 of the V2 scan stack, stacked on #10551. Same change as #10547, moved onto the rebased stack.
Stack:
vortex-layout(feat(layout): planner-driven V2 scan over physical plans #10550)vortex-file, DataFusion and DuckDB (feat(scan): select the V2 scan from vortex-file, DataFusion and DuckDB #10551)Changes
Only
vortex-layout/src/scan/v2/splits.rschanges.cut_atscanned every chunk boundary on each call, even for a small subrange. It now binary-searches the first boundary afterrows.startand stops atrows.end, so search work is O(log N + K), where N is the full boundary count and K is the count inside the range. Minimum split-size and final-sliver handling are unchanged.Checks run:
cargo clippy -p vortex-io -p vortex-scan -p vortex-layout -p vortex-file -p vortex-datafusion --all-targets --all-features -- -D warningscargo test -p vortex-layout --lib, with and withoutVORTEX_SCAN_V2=1🤖 Generated with Claude Code
https://claude.ai/code/session_01Hai26WGsNERxRHXpA3JboE
Generated by Claude Code