Skip to content

single_distinct_to_groupby not applied to DataFrame API count(DISTINCT) #23401

Description

@fangchenli

Describe the bug

The SingleDistinctToGroupBy optimizer rule rewrites a grouped count(DISTINCT col) into a cheaper nested double aggregate. It fires for plans built by the SQL planner but not for the logically-equivalent plan built by the DataFrame API, so the DataFrame path runs materially slower for the same query.

The divergence is caused by how each front-end places the output alias:

  • SQL (count(DISTINCT v) AS n): the alias becomes an outer Projection; Aggregate.aggr_expr holds a bare Expr::AggregateFunction.
  • DataFrame API (count(col("v")).distinct().alias("n")): the alias wraps the aggregate directly, so aggr_expr holds Expr::Alias(AggregateFunction).

is_single_distinct_agg only matches a bare Expr::AggregateFunction and returns false for the aliased shape, so the rule never fires for the DataFrame API.

To Reproduce

No response

Expected behavior

No response

Additional context

No response

Activity

  1. added theissue type on Jul 9, 2026
  2. added 5 commits that reference this issue on Jul 9, 2026
    de18cf2
    1aa3e4c
    811088a
    c388b94
    7975288
  3. qazxcdswe123 commented on Jul 9, 2026

    @qazxcdswe123
    Contributor

    I think this might be relavant #1468? (But not helpful though)

  4. fangchenli commented on Jul 9, 2026

    @fangchenli
    Author

    I think this might be relavant #1468? (But not helpful though)

    Yes, related. But #1468 has a bigger scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions