Skip to content

Cherry-pick apache/datafusion#25019: ignore constant aggregate completion boundaries - #203

Closed
Tristan1900 wants to merge 1 commit into
wenqi.mou/cherry-pick/apache-pr-25072-20261111from
wenqi.mou/cherry-pick/apache-pr-25019-20261111
Closed

Tristan1900 wants to merge 1 commit into
wenqi.mou/cherry-pick/apache-pr-25072-20261111from
wenqi.mou/cherry-pick/apache-pr-25019-20261111

Conversation

@Tristan1900

@Tristan1900 Tristan1900 commented Nov 11, 2026 •

Copy link
Copy Markdown
  • Backport apache/datafusion#25019 to exclude constant group keys from aggregate completion boundaries.
  • Preserve incremental aggregation when all remaining group keys are ordered. Depends on #202.

@codecov-commenter

codecov-commenter commented Nov 11, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.92683% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.21%. Comparing base (eda99c6) to head (f056cb6).

Files with missing lines Patch % Lines
datafusion/physical-plan/src/aggregates/mod.rs 82.92% 2 Missing and 12 partials ⚠️
Additional details and impacted files
@@                                Coverage Diff                                 @@
##           wenqi.mou/cherry-pick/apache-pr-25072-20261111     #203      +/-   ##
==================================================================================
- Coverage                                           81.21%   81.21%   -0.01%     
==================================================================================
  Files                                                1111     1111              
  Lines                                              388447   388527      +80     
  Branches                                           388447   388527      +80     
==================================================================================
+ Hits                                               315492   315550      +58     
- Misses                                              54357    54368      +11     
- Partials                                            18598    18609      +11     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Tristan1900
Tristan1900 marked this pull request as ready for review November 12, 2026 18:23
@Tristan1900
Tristan1900 force-pushed the wenqi.mou/cherry-pick/apache-pr-25072-20261111 branch from a2161fd to efed976 Compare November 12, 2026 18:36
@Tristan1900
Tristan1900 force-pushed the wenqi.mou/cherry-pick/apache-pr-25019-20261111 branch from 368b632 to 70c3f25 Compare November 12, 2026 18:36
## Which issue does this PR close?

- Closes apache#25018

## Rationale for this change

A grouping column with one fixed value never changes, so it cannot tell
aggregation that an earlier group is complete.
Currently the logic uses such columns when choosing how aggregation
processes its input. This can select the ordered path even when the
remaining grouping columns are not ordered.

## What changes are included in this PR?

- Ignore fixed grouping columns when deciding when groups can be
completed.
- Preserve ordered aggregation when all grouping columns that can change
are ordered.
- Update relevant plans

## What is the testing strategy for this PR?

Added tests for a grouping column fixed by a filter. The tests fail
without the fix.

## Are there any user-facing changes?

No query results or public APIs change. DataFusion now chooses the
appropriate aggregation path for fixed grouping columns.

(cherry picked from commit 6ff484a)
@Tristan1900
Tristan1900 force-pushed the wenqi.mou/cherry-pick/apache-pr-25072-20261111 branch from efed976 to eda99c6 Compare November 12, 2026 18:56
@Tristan1900
Tristan1900 force-pushed the wenqi.mou/cherry-pick/apache-pr-25019-20261111 branch from 70c3f25 to f056cb6 Compare November 12, 2026 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants