You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
String --> TableReference parsing does not properly handle " and . #4532
Describe the bug
Some DataFusion APIs take a string and convert to a table name but do not parse the identifiers correctly (specifically they ignore case normalization and quotes
An interesting problem I've found is that since the method takes a &str and returns a TableReference bounded by the lifetime of that &str, it can't mutate it (well, allocate a new String based off of it). This raises the question of how this identifier would be parsed:
ctx.register_table("\"foo\"\".bar\"", t);
Which I believe should register a table foo".bar
So it doesn't seem possible for current TableReference to handle this case as would need to allocate a new String to replace that inner "" to just "
It seems might be necessary to change TableReference to have Cow<'a, str> for its fields instead of raw &'a str's but this affects the rest of the code base. Will give it a shot to see how it goes.
Edit: also your example about case normalization should be
ctx.register_table("Bar", t);
instead, as being quoted would actually make it case-sensitive.
It seems might be necessary to change TableReference to have Cow<'a, str> for its fields instead of raw &'a str's but this affects the rest of the code base. Will give it a shot to see how it goes.
Activity
Jefffrey commentedon Feb 3, 2023
So the method of interest is here:
https://github.com/apache/arrow-datafusion/blob/1ec2a839645013cc9a2a4f78d8cadcec44d865e2/datafusion/common/src/table_reference.rs#L180-L196
An interesting problem I've found is that since the method takes a
&strand returns aTableReferencebounded by the lifetime of that&str, it can't mutate it (well, allocate a newStringbased off of it). This raises the question of how this identifier would be parsed:Which I believe should register a table
foo".barSo it doesn't seem possible for current
TableReferenceto handle this case as would need to allocate a newStringto replace that inner""to just"It seems might be necessary to change
TableReferenceto haveCow<'a, str>for its fields instead of raw&'a str's but this affects the rest of the code base. Will give it a shot to see how it goes.Edit: also your example about case normalization should be
instead, as being quoted would actually make it case-sensitive.
alamb commentedon Feb 3, 2023
A Cow would make sense to me. Maybe we could then remove
OwnedTableReferencehttps://docs.rs/datafusion-common/17.0.0/datafusion_common/enum.OwnedTableReference.html 🤔