Skip to content

JSON version of display_indent() #2889

Closed

Description

@avantgardnerio

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

A significant portion of my time on #2885 was spent manually formatting subqueries like:

Projection: #SUM(lineitem.l_extendedprice) / Float64(7) AS avg_yearly
  Aggregate: groupBy=[[]], aggr=[[SUM(#lineitem.l_extendedprice)]]
    Filter: #part.p_brand = Utf8("Brand#23") AND #part.p_container = Utf8("MED BOX") AND #lineitem.l_quantity < (Subquery: Projection: Float64(0.2) * #AVG(lineitem.l_quantity)
  Aggregate: groupBy=[[]], aggr=[[AVG(#lineitem.l_quantity)]]
    Filter: #lineitem.l_partkey = #part.p_partkey
      TableScan: lineitem)
      Inner Join: #lineitem.l_partkey = #part.p_partkey
        TableScan: lineitem
        TableScan: part

into

Projection: #SUM(lineitem.l_extendedprice) / Float64(7) AS avg_yearly
  Aggregate: groupBy=[[]], aggr=[[SUM(#lineitem.l_extendedprice)]]
    Filter: #part.p_brand = Utf8("Brand#23") AND #part.p_container = Utf8("MED BOX") 
        AND #lineitem.l_quantity < (
            Subquery: Projection: Float64(0.2) * #AVG(lineitem.l_quantity)
                Aggregate: groupBy=[[]], aggr=[[AVG(#lineitem.l_quantity)]]
                    Filter: #lineitem.l_partkey = #part.p_partkey
                        TableScan: lineitem
        )
      Inner Join: #lineitem.l_partkey = #part.p_partkey
        TableScan: lineitem
        TableScan: part

because the indents aren't trustworthy when subqueries are taken into account.

Also, I've had a hard time writing unit tests for optimizers because building them programmatically is verbose and error-prone.

Describe the solution you'd like

If we were to create a visitor method like display_json(), and an equivalent LogicalPlan::from_json() then both problems would be solved fairly elegantly. As a bonus, plans would be readable and manipulable by common off the the shelf tools and readable by the uninitiated.

Describe alternatives you've considered

Fixing the indentation on the formatter, writing different types of tests, crying into my beer.

Activity

  1. avantgardnerio commented on Jul 12, 2022

    @avantgardnerio
    ContributorAuthor

    As a potential implementation option, we could use something like https://docs.serde.rs/serde_json/ with our existing visitor pattern. I know some folks are reticent to add dependencies though, so I thought I would open this up for discussion. I'm after the ends, the means I'm flexible on.

  2. avantgardnerio commented on Jul 17, 2022

    @avantgardnerio
    ContributorAuthor

    Thank you!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions