You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@alamb@jonahgao
I have already made some progress on this part. If you don't mind, I would like to finish it. I want to use this update to become more familiar with DataFusion's code, and I would greatly appreciate any guidance you can offer.
Here are some sqlparser Pull Requests that involve modifications to DataFusion, which you can use as references:
Since sqlparser now differentiates between INNER JOIN and JOIN, the parsed result no longer matches the original test cases. For example:
TestStatementWithDialect{sql:" SELECT j1_string, j2_string FROM ( SELECT distinct j1_id, j1_string, j2_string from j1 INNER join j2 ON j1.j1_id = j2.j2_id order by j1.j1_id desc limit 10 ) abc ORDER BY abc.j2_string",expected:r#"SELECT abc.j1_string, abc.j2_string FROM (SELECT DISTINCT j1.j1_id, j1.j1_string, j2.j2_string FROM j1 JOIN j2 ON (j1.j1_id = j2.j2_id) ORDER BY j1.j1_id DESC NULLS FIRST LIMIT 10) AS abc ORDER BY abc.j2_string ASC NULLS LAST"#,parser_dialect:Box::new(GenericDialect{}),unparser_dialect:Box::new(UnparserDefaultDialect{}),},
Should the JOIN in expected be changed to INNER JOIN?
If we make this distinction, it will affect printing and the unparser, which will indirectly impact many test cases. Do you have any good suggestions?
Activity
jonahgao commentedon Mar 8, 2025
take
PokIsemaine commentedon Mar 8, 2025
@alamb @jonahgao
I have already made some progress on this part. If you don't mind, I would like to finish it. I want to use this update to become more familiar with DataFusion's code, and I would greatly appreciate any guidance you can offer.
Here are some sqlparser Pull Requests that involve modifications to DataFusion, which you can use as references:
Changelog: https://github.com/apache/datafusion-sqlparser-rs/blob/main/changelog/0.55.0.md
OrderBy: apache/datafusion-sqlparser-rs#1724
SQLExpr::Case: apache/datafusion-sqlparser-rs#1733
ColumnDef Collation: apache/datafusion-sqlparser-rs#1731
DataType: apache/datafusion-sqlparser-rs#1739
EXECUTE: apache/datafusion-sqlparser-rs#1717
CreateTable: apache/datafusion-sqlparser-rs#1664
QualifiedWildcard apache/datafusion-sqlparser-rs#1680
TypedString: apache/datafusion-sqlparser-rs#1739
SelectFlavor: apache/datafusion-sqlparser-rs#1713
StartTransaction: apache/datafusion-sqlparser-rs#1718
jonahgao commentedon Mar 8, 2025
Thank you @PokIsemaine for your work. Assigned this issue to you ❤️
PokIsemaine commentedon Mar 9, 2025
I want to discuss
JoinType:JoinOperator
Do we need to distinguish between
Inner/Join,Left/LeftOuter, andRight/RightOuterinJoinType, like this?Since
sqlparsernow differentiates betweenINNER JOINandJOIN, the parsed result no longer matches the original test cases. For example:Should the
JOINinexpectedbe changed toINNER JOIN?If we make this distinction, it will affect printing and the unparser, which will indirectly impact many test cases. Do you have any good suggestions?
jonahgao commentedon Mar 9, 2025
I think we don't need to distinguish them as they are equivalent. Introducing new join type variants would result in more significant changes.