Skip to content

Lower mixed DataFusion expressions with native subtree extraction - #11669

Draft
danking wants to merge 2 commits into
dk/df-expression-prerequisitesfrom
dk/df-expression-consumer
Draft

danking wants to merge 2 commits into
dk/df-expression-prerequisitesfrom
dk/df-expression-consumer

Conversation

@danking

@danking danking commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Build mixed Vortex/DF expression trees for scan filters and projections. Compatible operators remain native, unsupported fragments share typed DF callbacks, and conditional evaluation, Scalar-versus-Array shape, UDF-visible fields, and numeric behavior are preserved.

Prerequisites, reviewed independently:

The temporary review base dk/df-expression-prerequisites combines those two prerequisites so this diff contains only their consumer. Keep this PR draft until both land; then rebase and retarget to develop. Do not merge into the temporary review base.

The consumer overrides advisory conversion for whole positive optional filters, including partition-routing CASE/hash membership, while retaining DataFusion's exact enforcement and generation tracking. Seven focused mixed-expression tests and existing converter/scan tests cover the new behavior. Existing code keeps its original import/name style; unrelated qualification churn was removed.

Before merge: eliminate per-node return_field calls that repeatedly format whole subtrees. Memoization alone does not establish linear planning complexity.

Replaces the consumer portion of #11432. The original implementation had targeted checks; no builds, tests, or formatters were rerun for this source-reviewed split. Prepared with Codex; split commits reviewed in the designated review chat. @danking

Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant