You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Optimising an already optimised logical plan results in redundant pushed down filters #408
If you run a LogicalPlan that has a TableProviderFilterPushDownTableProvider, then the filter pushdown optimiser rule will keep adding filters to the pushed down table scan without checking if they're already present.
Given the plan:
Filter: #a Eq Int64(1)
TableScan: projection=None
it will be optimised to the following plan the first time it runs through the filter:
Run a logical plan that pushes predicates down to the table provider through the plan optimiser twice. All pushed down expression filters will be duplicated in the table scan filters.
Expected behavior
The plan should not change if it's ran through the optimiser multiple times.
Additional context
Whilst the current behaviour produces correct plans they are not optimal because they can contain redunant duplicate filters if they're run through the optimiser multiple times.
Activity