Repository navigation
Parse identifiers properly for TableReferences - #5183
Conversation
| 1 => Self::Bare { | ||
| table: parts.remove(0).into(), | ||
| }, | ||
| 2 => Self::Partial { | ||
| schema: parts[0], | ||
| table: parts[1], | ||
| schema: parts.remove(0).into(), | ||
| table: parts.remove(0).into(), | ||
| }, | ||
| 3 => Self::Full { | ||
| catalog: parts[0], | ||
| schema: parts[1], | ||
| table: parts[2], | ||
| catalog: parts.remove(0).into(), | ||
| schema: parts.remove(0).into(), | ||
| table: parts.remove(0).into(), | ||
| }, | ||
| _ => Self::Bare { table: s }, | ||
| // TODO: should normalize? | ||
| _ => Self::Bare { table: s.into() }, |
There was a problem hiding this comment.
parse_identifiers will return String anyway (wrapped in Ident) so only this last case could actually be Cow::Borrowed(...)
also unsure about how to handle this last case, i followed in footsteps of what was done before (just taking entire string), since alternative would be to panic which seems undesirable. also unsure if desirable to normalize it or not
There was a problem hiding this comment.
I think documenting the behavior is about the best we can do in this case.
| .map(|id| match id.quote_style { | ||
| Some(_) => id.value, | ||
| // TODO: someway to be able to toggle this functionality? | ||
| None => id.value.to_ascii_lowercase(), | ||
| }) |
There was a problem hiding this comment.
comparing with datafusion-sql package where some functionality is duplicated:
unsure if also should have a way to toggle normalization, as for datafusion-sql package it allows it:
but since this parse_str is meant to be used to implement From<&str>, there wouldn't really be a way to pass information on whether to toggle or not
There was a problem hiding this comment.
I agree -- if users want to normalize their identifiers I think they'll have to do it manually.
| } | ||
|
|
||
| /// Parse a string into a TableReference, by splittig on `.` | ||
| fn parse_identifiers(s: &str) -> Result<Vec<Ident>> { |
There was a problem hiding this comment.
could maybe upstream this whole function into sqlparser-rs instead?
There was a problem hiding this comment.
That certainly seems like a good idea (at least to file a ticket to track)
|
@alamb you mentioned possibility of removing however even though using |
|
I think CI is failing due to an issue fixed in #5177 |
I think that makes sense for this PR. Thanks @Jefffrey |
| .map(|id| match id.quote_style { | ||
| Some(_) => id.value, | ||
| // TODO: someway to be able to toggle this functionality? | ||
| None => id.value.to_ascii_lowercase(), | ||
| }) |
There was a problem hiding this comment.
I agree -- if users want to normalize their identifiers I think they'll have to do it manually.
| /// See more detail in <https://github.com/apache/arrow-datafusion/issues/4532> | ||
| /// Will normalize (convert to lowercase) any unquoted identifiers. | ||
| /// e.g. `Foo` will be parsed as `foo`, and `"Foo"".bar"` will be parsed as | ||
| /// `Foo".bar` (note the preserved case and requiring two double quotes to represent |
There was a problem hiding this comment.
| /// `Foo".bar` (note the preserved case and requiring two double quotes to represent | |
| /// `Foo.bar` (note the preserved case and requiring two double quotes to represent |
There was a problem hiding this comment.
It needs to contain the single double quote since it was escaped by prepending it with another double quote in the original, unless you mean to change the example?
There was a problem hiding this comment.
No, I think I was confused. Sorry about that
| 1 => Self::Bare { | ||
| table: parts.remove(0).into(), | ||
| }, | ||
| 2 => Self::Partial { | ||
| schema: parts[0], | ||
| table: parts[1], | ||
| schema: parts.remove(0).into(), | ||
| table: parts.remove(0).into(), | ||
| }, | ||
| 3 => Self::Full { | ||
| catalog: parts[0], | ||
| schema: parts[1], | ||
| table: parts[2], | ||
| catalog: parts.remove(0).into(), | ||
| schema: parts.remove(0).into(), | ||
| table: parts.remove(0).into(), | ||
| }, | ||
| _ => Self::Bare { table: s }, | ||
| // TODO: should normalize? | ||
| _ => Self::Bare { table: s.into() }, |
There was a problem hiding this comment.
I think documenting the behavior is about the best we can do in this case.
| } | ||
|
|
||
| /// Parse a string into a TableReference, by splittig on `.` | ||
| fn parse_identifiers(s: &str) -> Result<Vec<Ident>> { |
There was a problem hiding this comment.
That certainly seems like a good idea (at least to file a ticket to track)
|
Benchmark runs are scheduled for baseline = 0dfc66d and contender = 3f9b996. 3f9b996 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Closes #4532
Rationale for this change
Be able to parse identifiers properly for TableReference:
What changes are included in this PR?
Change TableReference to hold Cows of its str's since need to allocate new String due to parsing (not to mention for normalization as well)
Use SQL parser to parse the possible multipart identifier, which handles quoted identifiers correctly
Are these changes tested?
New unit test added
Are there any user-facing changes?
TableReference public API is changed (
Cow<str>instead of raw&str)