Repository navigation
Add support for UNION [DISTINCT] sql - #1068
Conversation
|
Use the way mentioned by @Dandandan @alamb, |
19bf016 to
6f75105
Compare
| } | ||
| _ => Err(DataFusionError::NotImplemented(format!( | ||
| "Only UNION ALL is supported, found {}", | ||
| "Only UNION ALL and UNION are supported, found {}", |
There was a problem hiding this comment.
| "Only UNION ALL and UNION are supported, found {}", | |
| "Only UNION ALL and UNION [DISTINCT] are supported, found {}", |
| let err = logical_plan(sql).expect_err("query should have failed"); | ||
| assert_eq!( | ||
| "NotImplemented(\"Only UNION ALL is supported, found EXCEPT\")", | ||
| "NotImplemented(\"Only UNION ALL and UNION are supported, found EXCEPT\")", |
There was a problem hiding this comment.
| "NotImplemented(\"Only UNION ALL and UNION are supported, found EXCEPT\")", | |
| "NotImplemented(\"Only UNION ALL and UNION [DISTINCT] are supported, found EXCEPT\")", |
|
This looks good @xudong963 thank you! One suggestion I have is to also test the |
| } | ||
|
|
||
| #[tokio::test] | ||
|
|
There was a problem hiding this comment.
nitpick, to keep the style consistent, we typically don't leave a newline between #[tokio::test] and the test implementation.
ec10910 to
332e472
Compare
|
all comments have fixed @Dandandan @houqp cc @alamb |
alamb
left a comment
There was a problem hiding this comment.
This is a pretty awesome PR @xudong963 -- thank you! I really like how the support just sort of comes together from the existing parts and pieces
| /// apply union distinct | ||
| pub fn union_distinct(&self) -> Result<Self> { |
There was a problem hiding this comment.
| /// apply union distinct | |
| pub fn union_distinct(&self) -> Result<Self> { | |
| /// Apply deduplication: Only distinct (different) values are returned) | |
| pub fn distinct(&self) -> Result<Self> { |
I suggest renaming this to just distinct as there is only one input, so I find the use of the term union somewhat confusing
| let left_plan = self.set_expr_to_plan(left.as_ref(), None, ctes)?; | ||
| let right_plan = self.set_expr_to_plan(right.as_ref(), None, ctes)?; | ||
| let union_plan = union_with_alias(left_plan, right_plan, alias)?; | ||
| LogicalPlanBuilder::from(union_plan) |
| const QUERY: &str = | ||
| "SELECT customer_id, revenue FROM sales ORDER BY revenue DESC limit 3"; | ||
|
|
||
| const QUERY2: &str = "SELECT customer_id, revenue FROM sales UNION SELECT customer_id, revenue FROM sales ORDER BY revenue DESC limit 3"; |
There was a problem hiding this comment.
Can you please move this test into tests/sql.rs? I don't think UNION has anything to do with user defined plans.
alamb
left a comment
There was a problem hiding this comment.
I think renaming the function on LogicalPlanBuilder and moving the test is all I think is necessary prior to merging this PR.
332e472 to
d6647b3
Compare
d6647b3 to
68277da
Compare
|
Thanks for your great comments which exactly improve my engineering ability! @alamb @houqp @Dandandan all comments have fixed @alamb |
Yes, me too. The process helps me know well existing parts! |
|
Really nice work @xudong963 ! |
|
Thanks @xudong963 ! 👍 |
Which issue does this PR close?
Closes #998
Earlier version #1029
Rationale for this change
What changes are included in this PR?
Are there any user-facing changes?