Skip to content

Reject get_field pushdown when its source cannot convert - #10083

Merged
robert3005 merged 1 commit into
vortex-data:developfrom
RainyPixel:fix/get-field-pushdown
Sep 29, 2026
Merged

robert3005 merged 1 commit into
vortex-data:developfrom
RainyPixel:fix/get-field-pushdown

Conversation

@RainyPixel

Copy link
Copy Markdown
Contributor

Problem

can_scalar_fn_be_pushed_down accepts every GetFieldFunc without checking its source. get_field(named_struct(...), 'value') is valid in DataFusion, but named_struct has no Vortex expression conversion. Marking the whole expression pushable lets planning proceed into a conversion error instead of leaving it above the scan.

Change

Only mark GetFieldFunc pushable when its complete expression converts. The regression builds the actual DataFusion named_struct and get_field physical expressions and checks the pushdown decision. It fails on current develop and passes with this change.

Verification

  • cargo test -p vortex-datafusion test_get_field_of_unsupported_struct_function_stays_in_datafusion --lib (with the repository-pinned flatc 25.12.19)
  • cargo +nightly fmt --package vortex-datafusion -- --check

#9785 has a broader schema-aware expression-conversion redesign that also addresses this class of error. This PR offers a focused fix and regression for current develop; it can be superseded if #9785 lands first.

@robert3005 robert3005 added the changelog/fix A bug fix label Sep 27, 2026
@codspeed

codspeed Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 1 benchmark measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. This result is not comparable, so it counts as unchanged.

Preventing compiler optimizations

✅ 2062 untouched benchmarks
⏩ 503 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚠️ Simulation bench_compare_sliced_dict_primitive[(3333, 10000)] 80.3 µs < 1 ns N/A

Comparing RainyPixel:fix/get-field-pushdown (aff23f1) with develop (50c3f9f)

Open in CodSpeed

Footnotes

  1. 503 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Signed-off-by: Ivan Bobchenkov <me@bobchenkov.ru>
@RainyPixel
RainyPixel force-pushed the fix/get-field-pushdown branch from 38ab9e6 to aff23f1 Compare September 29, 2026 09:20
@robert3005
robert3005 merged commit 566f82b into vortex-data:develop Sep 29, 2026
128 of 130 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants