Repository navigation
Reject get_field pushdown when its source cannot convert - #10083
Merged
robert3005 merged 1 commit intoSep 29, 2026
Merged
Conversation
robert3005
approved these changes
Sep 27, 2026
Merging this PR will not alter performance
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| Simulation | bench_compare_sliced_dict_primitive[(3333, 10000)] |
80.3 µs | < 1 ns | N/A |
Comparing RainyPixel:fix/get-field-pushdown (aff23f1) with develop (50c3f9f)
Footnotes
-
503 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Signed-off-by: Ivan Bobchenkov <me@bobchenkov.ru>
RainyPixel
force-pushed
the
fix/get-field-pushdown
branch
from
September 29, 2026 09:20
38ab9e6 to
aff23f1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
can_scalar_fn_be_pushed_downaccepts everyGetFieldFuncwithout checking its source.get_field(named_struct(...), 'value')is valid in DataFusion, butnamed_structhas no Vortex expression conversion. Marking the whole expression pushable lets planning proceed into a conversion error instead of leaving it above the scan.Change
Only mark
GetFieldFuncpushable when its complete expression converts. The regression builds the actual DataFusionnamed_structandget_fieldphysical expressions and checks the pushdown decision. It fails on currentdevelopand passes with this change.Verification
cargo test -p vortex-datafusion test_get_field_of_unsupported_struct_function_stays_in_datafusion --lib(with the repository-pinnedflatc25.12.19)cargo +nightly fmt --package vortex-datafusion -- --check#9785 has a broader schema-aware expression-conversion redesign that also addresses this class of error. This PR offers a focused fix and regression for current
develop; it can be superseded if #9785 lands first.