Skip to content

fix: Remove !=0 check from supports_collect_by_thresholds - #20730

Merged
alamb merged 5 commits into
apache:mainfrom
jonathanc-n:remove-threshold-todo
Mar 11, 2026
Merged

alamb merged 5 commits into
apache:mainfrom
jonathanc-n:remove-threshold-todo

Conversation

@jonathanc-n

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

Small change to remove the TODO + != 0 check. Since the commit was made two years ago, statistics have been changed to use Precision. Exact(0) and inexact(0) are now valid stats while absent will evaluate to false

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions Bot added the optimizer Optimizer rules label Mar 5, 2026
@jonathanc-n

Copy link
Copy Markdown
Contributor Author

cc @gabotechs

@github-actions github-actions Bot added the sqllogictest SQL Logic Tests (.slt) label Mar 5, 2026
Co-authored-by: Daniël Heres <danielheres@gmail.com>

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jonathanc-n

// them as unknown, `Absent` will return None (this is in regards to why
// `!=0` is not checked)
if let Some(byte_size) = stats.total_byte_size.get_value() {
*byte_size != 0 && *byte_size < threshold_byte_size

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when would byte size ever be zero ? This seems like if the size was zero something is wrong 🤔

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Byte size can be zero if working on empty table/partition or if filter exec filters down to zero rows.

This PR just removes this check as it is not longer a problem due to the addition of Precision (this check was added 2 years ago). This sanity check was previously here because getting statistics with limit would unwrap to zero (which would now be Absent)

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jonathanc-n

@alamb
alamb added this pull request to the merge queue Mar 11, 2026
Merged via the queue into apache:main with commit 129c58f Mar 11, 2026
34 checks passed
de-bgunter pushed a commit to de-bgunter/datafusion that referenced this pull request Mar 24, 2026
…#20730)

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes apache#123` indicates that this PR will close issue apache#123.
-->

- Closes #.

## Rationale for this change
Small change to remove the TODO + != 0 check. Since the commit was made
two years ago, statistics have been changed to use Precision. Exact(0)
and inexact(0) are now valid stats while absent will evaluate to false
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

---------

Co-authored-by: Daniël Heres <danielheres@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants