Skip to content

Cherry-pick apache/datafusion#25072: clear ordering for unordered aggregation - #202

Closed
Tristan1900 wants to merge 2 commits into
wenqi.mou/cherry-pick/apache-pr-25706-20261112from
wenqi.mou/cherry-pick/apache-pr-25072-20261111
Closed

Tristan1900 wants to merge 2 commits into
wenqi.mou/cherry-pick/apache-pr-25706-20261112from
wenqi.mou/cherry-pick/apache-pr-25072-20261111

Conversation

@Tristan1900

@Tristan1900 Tristan1900 commented Nov 11, 2026 •

Copy link
Copy Markdown
  • Stop unordered aggregation, including PartialReduce, from advertising its input's ordering. Backport apache/datafusion#25072.
  • Keep the prerequisite aggregate-mode guard from apache/datafusion#24961 in a separate compatibility commit.
  • Depends on #210, which restores the CLI storage-test image.

@codecov-commenter

codecov-commenter commented Nov 11, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.05128% with 7 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (wenqi.mou/cherry-pick/apache-pr-25706-20261112@45c4b91). Learn more about missing BASE report.

Files with missing lines Patch % Lines
datafusion/physical-plan/src/aggregates/mod.rs 82.05% 2 Missing and 5 partials ⚠️
Additional details and impacted files
@@                                Coverage Diff                                @@
##             wenqi.mou/cherry-pick/apache-pr-25706-20261112     #202   +/-   ##
=================================================================================
  Coverage                                                  ?   81.21%           
=================================================================================
  Files                                                     ?     1111           
  Lines                                                     ?   388447           
  Branches                                                  ?   388447           
=================================================================================
  Hits                                                      ?   315492           
  Misses                                                    ?    54357           
  Partials                                                  ?    18598           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Tristan1900
Tristan1900 marked this pull request as ready for review November 12, 2026 18:23
@Tristan1900
Tristan1900 force-pushed the wenqi.mou/cherry-pick/apache-pr-25072-20261111 branch from a2161fd to efed976 Compare November 12, 2026 18:36
@Tristan1900
Tristan1900 changed the base branch from branch-55 to wenqi.mou/cherry-pick/apache-pr-25706-20261112 November 12, 2026 18:37
Tristan1900 and others added 2 commits November 12, 2026 13:55
Grouping sets can change group keys, and partial reduction does not
recognize completed groups. Keep both in linear mode before changing
how aggregate output ordering is propagated.

Port only the aggregate-mode guard from apache#24961;
retain DataFusion 55's existing aggregation streams.

Partial backport of 5b389eb.
## Which issue does this PR close?

N/A

## Rationale for this change
Hash Aggregate Partial reduce propagate the ordering of the child even
though it does not maintain child ordering

## What changes are included in this PR?
clear ordering and test

## What is the testing strategy for this PR?
unit test

## Are there any user-facing changes?
not really

(cherry picked from commit bb7d9f0)
@Tristan1900
Tristan1900 force-pushed the wenqi.mou/cherry-pick/apache-pr-25072-20261111 branch from efed976 to eda99c6 Compare November 12, 2026 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants