Skip to content

Convert Expr to a parsable representation #7165

Open

Description

@Blajda

Is your feature request related to a problem or challenge?

In the delta-rs project we support operations such as delete, update, and merge where users can supply predicate as either a string or a DataFusion Expr. String predicates go through sql-parser to obtain an Expr and are evaluated. At the end of each operation the expression must be converted back to a string to store in the transaction log for conflict resolution.

The implementations for create_name and canonical_name almost fit this need but scalar values are surrounded by their type which cannot be parsed by sql-parser.

E.G col1 = 1 becomes col1 = Int32(1)

Describe the solution you'd like

Given an Expr one should be able to obtain it's string representation that can be parsed by sql parser.

Describe alternatives you've considered

No response

Additional context

No response

Activity

  1. parkma99 commented on Aug 1, 2023

    @parkma99
    Contributor

    I made a PR #6708 about printing Expr two weeks ago. Could you give more infomation? Thanks , and I will work it this night.

  2. Blajda commented on Aug 1, 2023

    @Blajda
    Author

    Thanks @parkma99

    On delta-rs we this method to convert an sql expression to Expr.
    We can almost convert it back to a sql string using canonical_name however it fails to parse using the above code.
    This is because the method uses Debug format for scalar values which results in output like Int32(1). The parser thinks Int32(1) is a function.

    I would like to have a function like canonical_name except it format scalar values as their display value.

  3. parkma99 commented on Aug 1, 2023

    @parkma99
    Contributor

    cc @alamb. Do we need a new function to Convert Expr to a parsable representation? Or we just change the canonical_name function. Thanks 😊

  4. alamb commented on Aug 3, 2023

    @alamb
    Contributor

    We could potentially propose changing canonical_name - however as I recall that function is used to create the column names so it might have a large change.

    Another possibility, which is somewhat of a hack, might be to add a expr rewrite pass parsing stuff like c = Int32(1) that rewrites Function: int32(arg) --> ScalarValue::Int32

  5. parkma99 commented on Aug 4, 2023

    @parkma99
    Contributor

    Another possibility, which is somewhat of a hack, might be to add a expr rewrite pass parsing stuff like c = Int32(1) that rewrites Function: int32(arg) --> ScalarValue::Int32

    just like does??
    https://github.com/apache/arrow-datafusion/blob/02da0445f8c94019c1526d6b1759492fda266cdf/datafusion-examples/examples/rewrite_expr.rs#L163-L187

  6. alamb commented on Aug 4, 2023

    @alamb
    Contributor

    just like does??

    That is a good example of using the rewriter ! Though the actual rewrite rule is a little different

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions