Repository navigation
Prune zones beneath nullable structs - #11641
Closed
connortsui20 wants to merge 1 commit into
Closed
connortsui20 wants to merge 1 commit into
connortsui20 wants to merge 1 commit into
Conversation
connortsui20
force-pushed
the
ct/nullable-struct-zone-maps
branch
from
November 19, 2026 19:20
23a26b5 to
fe8e020
Compare
Fields beneath nullable structs had no zone maps, so filters read zones that bounds could discard. Bounds over stored values still cover visible rows, and a zero physical NaN count proves there are no visible NaNs. Use those facts in pruning plans while keeping exact filtering masked. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/nullable-struct-zone-maps
branch
from
November 19, 2026 22:41
fe8e020 to
01d0b9a
Compare
This was referenced Nov 20, 2026
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On
develop, the writer omits zone maps for fields beneath nullable structs. A filter such ass.x < 10therefore reads candidate zones even when a lower bound could rule them out.This PR writes stored bounds and float NaN counts for those fields in
vortex.zonedlayouts. Both the classic and V2 scans use these statistics to skip zones when they prove that the filter has no matches.spresents.x404050501000NULLThe stored minimum of
40rules outs.x < 10. If the hidden value were0, the scan would keep the zone and check the rows. The writer omits null counts because a stored count of zero would incorrectly skip this zone fors.x IS NULL. A stored NaN count of zero proves there are no visible NaNs because hidden values can only add NaNs.Writers that use the legacy
vortex.statsformat still omit zone maps beneath nullable structs. Related to #11160, which normalizes hidden child values before writing.Stacked on #11638
This change was implemented and reviewed with OpenAI Codex.