Skip to content

Lower mixed DataFusion expressions into Vortex scans - #11432

Closed
danking wants to merge 4 commits into
dk/df-expression-shimfrom
dk/df-expression-pushdown
Closed

danking wants to merge 4 commits into
dk/df-expression-shimfrom
dk/df-expression-pushdown

Conversation

@danking

@danking danking commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Split into separately reviewable replacements:

Unrelated name/import rewrites in existing code and tests were removed. Functional behavior and existing test coverage were retained. #11669 remains draft pending the two prerequisites and its documented planning-cost follow-up.

Prepared with Codex; replacement commits source-reviewed in the designated review chat. No builds or tests rerun for this reorganization.

@codspeed

codspeed Bot commented Nov 12, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 10.73%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 1 improved benchmark
✅ 1920 untouched benchmarks
⏩ 50 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Walltime decode_chunked_neon[i64, 524288] 15.5 µs 14 µs +10.73%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing dk/df-expression-pushdown (9d70d95) with dk/df-expression-shim (6e97fda)

Open in CodSpeed

Footnotes

  1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@robert3005 robert3005 added the action/bench-sql Run SQL benchmarks without Vortex Compact on this PR label Nov 13, 2026
@danking danking added the changelog/feature A new feature label Nov 13, 2026
@danking
danking added this pull request to stack #11653 November 19, 2026 19:18
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

action/bench-sql Run SQL benchmarks without Vortex Compact on this PR changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants