Repository navigation
Check extension claims in each function - #11600
Draft
connortsui20 wants to merge 10 commits into
Draft
connortsui20 wants to merge 10 commits into
connortsui20 wants to merge 10 commits into
Conversation
This was referenced Nov 18, 2026
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/ext-claims
branch
from
November 18, 2026 16:43
2fbac1e to
b27aa23
Compare
connortsui20
force-pushed
the
ct/ext-claim-checks
branch
from
November 18, 2026 16:43
4e2045b to
5dfd9ae
Compare
This was referenced Nov 18, 2026
Merged
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/ext-claim-checks
branch
from
November 18, 2026 20:58
3c81806 to
96e7ef0
Compare
This was referenced Nov 19, 2026
connortsui20
added a commit
that referenced
this pull request
Nov 20, 2026
- Tracking Issue: #11115 Functions use the storage of any extension type today, so comparing two `Json` values compares their strings. A type needs a way to promise that its storage answers equality or order correctly before a function can trust it. This PR adds only the declaration, and #11600 makes each function check it. `ExtVTable::claims` returns an opaque `Claims` value. A `&'static [Claim]` slice was the alternative, but the slice could disagree with `DType::holds` about `Eq`, it had several values for the same promise (`[Ord]` and `[Eq, Ord]`), and it could only name properties. Private fields leave room for a custom order key later. AI assisted with this change. --------- Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Functions ran on the storage of any extension type, so
json_a = json_bcompared strings and the minimum of a WKB column was its smallest byte sequence. Zone maps, Bloom filters, and engine statistics were then built from those answers. Each function now requires the claim it needs when it binds, so a missing claim is a type error. It checks the claim again before it unwraps storage, because a path that skips binding has to fail instead of answering from storage.VectorandFixedShapeTensorclaim onlyEq, because the lexicographic order of their storage isn't an order anyone would sort vectors by.Json, WKB, and the other types claim nothing, because one value can have several storage forms.DuckDB builds table filters for simple comparisons without asking the
pushdown_expressionhook, so the scan declines each column whose dtype lacksOrdthroughsupports_pushdown_type. Turning off table filter pushdown for the whole scan was the alternative, but that loses it for every other column too. Moving those filters above the scan exposed two latent bugs, in the multi-file reader's projection order and in join runtime filters, which are fixed here.Files written before this change can hold statistics that are now rejected, such as the min and max of a
Jsoncolumn. A zoned layout that holds one turns off pruning and reads only its data child, and the footer skips them. A zone map that names an aggregate of the removed storage adapter, such asvortex.is_sorted[vortex.timestamp]or the same form for a third-party type, also turns off pruning instead of failing to open. For review, read the commits one at a time, since each one covers one family of functions.Breaking change: comparisons,
between,list_contains, the aggregates that order or compare values, and Bloom filters reject extension types that lack the claim they need, such asJsonand WKB.ExtensionStoragePushdownandAggregateFnSession::register_storage_orderedare removed, because a type that claimsOrdgets the order aggregates without a registration. TheScalarandRootvariants ofBoundExpressionare private, so that no code can build a node that skipsreturn_dtype.Stacked on #11579.
AI assisted with this change.