You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Expose SessionContext.add_optimizer_rule for Python-defined logical optimizer rules #1574
SessionContext::add_optimizer_rule registers an Arc<dyn OptimizerRule> that runs during logical plan optimization. The Python bindings only expose remove_optimizer_rule, never the additive side. PR #1557 ultimately landed add_physical_optimizer_rule for the physical pipeline via FFI but did not address the logical pipeline.
Lets users add domain-specific logical rewrites (predicate normalization, redundant join elimination, scan pruning informed by external metadata) without forking DataFusion. Complements the existing remove_optimizer_rule to round out the surface.
Why deferred
Blocked upstream. As documented in PR #1557, DataFusion does not currently expose an FFI bridge for the logical OptimizerRule / AnalyzerRule traits, and there are no Python constructors for LogicalPlan node variants -- a pure-Python rule could observe plans but not transform them. The PR #1557 commit history shows an initial attempt at a Python-defined logical rule that was abandoned for this reason. This is filed for tracking; it should be revisited once upstream lands an FFI_OptimizerRule (mirroring FFI_PhysicalOptimizerRule).
Activity