You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Creating a view in a new schema then trying to querying that view using its qualified name results in an error. See below:
DataFusion CLI v22.0.0
❯ create schema s;
0 rows in set. Query took 0.007 seconds.
❯ create view s.v as (select 1 as a);
0 rows in set. Query took 0.004 seconds.
❯ select * from s.v;
Schema error: No field named s.v.a. Valid fields are "s.v".a.
I'm also not able to use fully qualified field names to select from this view either:
❯ select a from s.v;
Schema error: No field named s.v.a. Valid fields are "s.v".a.
❯ select s.v.a;
Schema error: No field named s.v.a.
❯ select "s.v".a;
Schema error: No field named s.v.a.
To Reproduce
Using datafusion-cli
DataFusion CLI v22.0.0
❯ create schema s;
0 rows in set. Query took 0.007 seconds.
❯ create view s.v as (select 1 as a);
0 rows in set. Query took 0.004 seconds.
❯ select * from s.v;
Schema error: No field named s.v.a. Valid fields are "s.v".a.
Expected behavior
I can run select * from s.v (along with the qualified alternatives).
Additional context
Also fails in DataFusion 21.1.0 & 21.0.0:
DataFusion CLI v21.1.0
❯ create schema s;
0 rows in set. Query took 0.007 seconds.
❯ create view s.v as select 1 as a;
0 rows in set. Query took 0.004 seconds.
❯ select * from s.v
;
Schema error: No field named "s"."v"."a". Valid fields are "s.v"."a".
Works as expected in 20.0.0
DataFusion CLI v20.0.0
❯ create schema s;
0 rows in set. Query took 0.008 seconds.
❯ create view s.v as select 1 as a;
0 rows in set. Query took 0.004 seconds.
❯ select * from s.v;
+---+
| a |
+---+
| 1 |
+---+
1 row in set. Query took 0.010 seconds.
Can see it still refers to specific catalog, schema, and table. Whereas the subquery alias SubqueryAlias: cat.sch.v has erased this since the alias is cat.sch.v as a whole
Activity
Jefffrey commentedon Apr 15, 2023
Looks to be due to the InlineTableScan analyzer rule, specifically:
https://github.com/apache/arrow-datafusion/blob/68bd113a74f045cd24d01418c92c83f2e3a18853/datafusion/optimizer/src/analyzer/inline_table_scan.rs#L56-L79
Where it converts view into a subquery alias, using the
table_nameas the subquery alias:https://github.com/apache/arrow-datafusion/blob/68bd113a74f045cd24d01418c92c83f2e3a18853/datafusion/optimizer/src/analyzer/inline_table_scan.rs#L76
But doesn't rewrite the parent projection to use this correct subquery alias.
So given input plan:
After the rule this becomes:
The top level projection,
Projection: cat.sch.v.c1, cat.sch.v.c2, cat.sch.v.c3, taking onlyc1, is better understood as:Can see it still refers to specific catalog, schema, and table. Whereas the subquery alias
SubqueryAlias: cat.sch.vhas erased this since the alias iscat.sch.vas a wholescsmithr commentedon Apr 15, 2023
This rule was in place before (but on the
Optimizer). I believe changes made in #5343 are what led to this.I opened up #6022 which makes subquery aliases a table reference which should fix this issue.