Skip to content

String --> TableReference parsing does not properly handle " and . #4532

Closed

Description

@alamb

Describe the bug
Some DataFusion APIs take a string and convert to a table name but do not parse the identifiers correctly (specifically they ignore case normalization and quotes

To Reproduce
Register a table with

https://docs.rs/datafusion/15.0.0/datafusion/execution/context/struct.SessionContext.html#method.register_table

ctx.register_table("\"foo.bar\"", t);

registers a table bar in schema foo

Likewise

ctx.register_table("\"Bar\"", t);

registers a table named Bar

In SQL

  • CREATE TABLE "foo.bar" registers a table foo.bar
  • CREATE TABLE Bar... registers a table bar

Expected behavior
The table should be registered parsed as a name foo.bar

Additional context
Found while working on #4530

Activity

  1. added
    bugSomething isn't working
    sqlSQL Planner
    and removed
    sqlSQL Planner
    on Dec 6, 2022
  2. Jefffrey commented on Feb 3, 2023

    @Jefffrey
    Contributor

    So the method of interest is here:

    https://github.com/apache/arrow-datafusion/blob/1ec2a839645013cc9a2a4f78d8cadcec44d865e2/datafusion/common/src/table_reference.rs#L180-L196

    An interesting problem I've found is that since the method takes a &str and returns a TableReference bounded by the lifetime of that &str, it can't mutate it (well, allocate a new String based off of it). This raises the question of how this identifier would be parsed:

    ctx.register_table("\"foo\"\".bar\"", t);

    Which I believe should register a table foo".bar

    So it doesn't seem possible for current TableReference to handle this case as would need to allocate a new String to replace that inner "" to just "

    It seems might be necessary to change TableReference to have Cow<'a, str> for its fields instead of raw &'a str's but this affects the rest of the code base. Will give it a shot to see how it goes.

    Edit: also your example about case normalization should be

    ctx.register_table("Bar", t);

    instead, as being quoted would actually make it case-sensitive.

  3. alamb commented on Feb 3, 2023

    @alamb
    ContributorAuthor

    It seems might be necessary to change TableReference to have Cow<'a, str> for its fields instead of raw &'a str's but this affects the rest of the code base. Will give it a shot to see how it goes.

    A Cow would make sense to me. Maybe we could then remove OwnedTableReference https://docs.rs/datafusion-common/17.0.0/datafusion_common/enum.OwnedTableReference.html 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions