You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
EnforceSorting resorts the inout of UnionExec unnecessarily #4943
I think the reason is that UnionExec doesn't implement fn maintains_input_order(&self) -> bool. But sometimes it maintains_input_order. If we implement fn maintains_input_order(&self) -> bool as below (Taken from self.output_ordering implementation). I think the problem will be solved.
fnmaintains_input_order(&self) -> bool{let first_input_ordering = self.inputs[0].output_ordering();// If the Union is not partition aware and all the input ordering spec strictly equal with the first_input_ordering// `UnionExec` maintains input order//// It might be too strict here in the case that the input ordering are compatible but not exactly the same.// For example one input ordering has the ordering spec SortExpr('a','b','c') and the other has the ordering// spec SortExpr('a'), It is safe to derive the out ordering with the spec SortExpr('a').
!self.partition_aware
&& first_input_ordering.is_some()
&& self.inputs.iter().map(|plan| plan.output_ordering()).all(|ordering| {
ordering.is_some()
&& sort_expr_list_eq_strict_order(
ordering.unwrap(),
first_input_ordering.unwrap(),)})}
Activity
alamb commentedon Jan 17, 2023
cc @mustafasrepo and @mingmwang who I think have expertise in this area
mustafasrepo commentedon Jan 17, 2023
I think the reason is that UnionExec doesn't implement
fn maintains_input_order(&self) -> bool. But sometimes it maintains_input_order. If we implementfn maintains_input_order(&self) -> boolas below (Taken fromself.output_orderingimplementation). I think the problem will be solved.alamb commentedon Jan 17, 2023
Thank you @mustafasrepo -- I will attempt to write a reproducer for this and try our your proposed fix
alamb commentedon Jan 17, 2023
proposed PR: #4946
mingmwang commentedon Jan 18, 2023
Nice !!
alamb commentedon Jan 18, 2023
Yes I am quite pleased with how sophisticated the sorting based optimizations are becoming