Repository navigation
Validate nullability against physical validity - #10225
Closed
connortsui20 wants to merge 1 commit into
Closed
connortsui20 wants to merge 1 commit into
connortsui20 wants to merge 1 commit into
Conversation
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Member
Author
|
So this PR was generated by codex. On the one hand this is probably not necessary. But on the other hand, it does seem weird that we just allow safe callers to supply whatever statistic they want even if it might be wrong? Do we want to build in some better guarantees here? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Superseded by the narrower VarBinView validity check in #10269. The broader validation policy is deferred.
Original PR description
Tracking Issue: #10177
Summary
With accurate cached statistics, the existing validity shortcuts are correct. This PR changes whether validation may rely on those statistics when safe callers can populate the cache independently of the data.
For example, a validity mask of
[false, true]with an incorrect cachedMin = trueis treated as all-valid. The regression deliberately supplies that incorrect hint. It does not demonstrate a bug in normal statistic computation or propagation.This proposed validation policy is split out of #10178 for separate review. Its need and cost should be assessed independently of replacing the statistics API. Open question: should these boundaries inspect the mask, or should the statistics API restrict how trusted facts are established?
Changes
Inspect the actual mask when converting nullable validity to non-nullable validity, converting a boolean array to a mask, and checking that a
MaskedArraychild has no nulls. Both safeMaskedArrayconstruction paths use the same policy.The regression covers incorrect cached minima in both directions. Establishing an unchecked constructor invariant is distinct from demonstrating that a wrong hint can cause memory unsafety, which this PR does not establish.