Repository navigation
Add generic dynamic expressions with generation tracking - #10865
Conversation
Extract the generic predicate source and per-generation bindings from #10665. Reuse native dynamic-expression update tracking for predicates whose declared inputs remain stable. Co-authored-by: Robert Kruszewski <github@robertk.io> Source: #10665 (19a26fd) Signed-off-by: Nicholas Gates <nick@nickgates.com>
Merging this PR will improve performance by 23.35%3 warnings
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
Bumps `vortex-datafusion` from 0.90.0 to 0.93.0. DataFusion stays at 55.2.0, which satisfies 0.93.0's `datafusion ^55.0.0` requirement. The release tag will be `0.93.0-55.2.0`. This picks up: - **0.91.0**: generic dynamic expressions with generation tracking (vortex-data/vortex#10865), live statistics proofs for them (#10869), skipping file filters already proven true (#10894), and pushing DataFusion dynamic filters through Scan API expression trees (#10895) - **0.92.0**: DataFusion dynamic filters pushed into V2 scans (#11215) - **0.93.0**: `array_transform` pushdown (#11031) Note: `scan_v2` still defaults to `false` in 0.93.0, so the V2-only dynamic filter path needs `OPTIONS (scan_v2 'true')` or `SET vortex.scan_v2 = true`. Checked locally: `cargo fmt --check` and `cargo check --locked` pass. The tests were not run locally; CI runs them. Supersedes #19. --- _Generated by [Claude Code](https://claude.ai/code/session_01UeniBCmx9HLCPXaaVKUcL2)_ Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com>
Extract the generic predicate source and per-generation bindings from #10665. Reuse native dynamic-expression update tracking for predicates whose declared inputs remain stable.
DataFusion provides entirely dynamic expression trees, while our existing logic only handles dynamic comparisons with dynamic scalars.
This PR adds initial support for dynamic expression trees.