Repository navigation
Replace extension examples with independent conformance tests - #11361
Merged
Merged
Conversation
19 of 57 tasks
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
5 times, most recently
from
November 9, 2026 22:12
fc51d62 to
f2987e6
Compare
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
from
November 10, 2026 14:52
f2987e6 to
73f31f1
Compare
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
2 times, most recently
from
November 10, 2026 16:16
f32781e to
73a471c
Compare
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
from
November 10, 2026 16:52
73a471c to
53f31ab
Compare
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
from
November 10, 2026 18:18
53f31ab to
d5c2d05
Compare
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
from
November 10, 2026 23:12
d5c2d05 to
9e3f744
Compare
connortsui20
marked this pull request as ready for review
November 11, 2026 13:26
Member
Author
|
I think I want to also delete a lot of the existing tests too |
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
from
November 12, 2026 15:36
9e3f744 to
f0887af
Compare
Merging this PR will not alter performance1 warning
Comparing Footnotes
|
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
2 times, most recently
from
November 12, 2026 15:51
8e2125f to
35520db
Compare
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
from
November 12, 2026 15:56
35520db to
0d6def1
Compare
connortsui20
marked this pull request as draft
November 12, 2026 16:18
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
2 times, most recently
from
November 16, 2026 19:09
1ec883e to
a65043b
Compare
connortsui20
marked this pull request as ready for review
November 16, 2026 20:36
connortsui20
marked this pull request as draft
November 16, 2026 20:37
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
from
November 16, 2026 22:02
fc848fb to
a22b2f7
Compare
connortsui20
changed the base branch from
develop
to
ct/spatial-filtered-list-export
November 16, 2026 22:02
connortsui20
marked this pull request as ready for review
November 16, 2026 22:07
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
from
November 17, 2026 12:43
a22b2f7 to
a3770f9
Compare
robert3005
approved these changes
Nov 17, 2026
connortsui20
enabled stacked auto-merge (squash)
November 17, 2026 14:46
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/arrow-roundtrip-properties
branch
from
November 17, 2026 15:00
a3770f9 to
fbae5ff
Compare
stacked auto-merge was automatically disabled
November 17, 2026 15:12
Replaced by a newer auto-merge request
connortsui20
enabled stacked auto-merge (squash)
November 17, 2026 15:12
stacked auto-merge was automatically disabled
November 17, 2026 15:27
Auto-merge was disabled for another pull request in the stack
connortsui20
added a commit
that referenced
this pull request
Nov 17, 2026
## Summary Tracking Issue: #11114. Builds on #11361. Known extension importers can silently fall back to plain storage when Arrow uses a different layout. For example, PyArrow marks list elements nullable, which can make a vector column lose its `Vector` type. Valid input needs normalization into the extension's declared storage, while invalid input needs an error. ## Changes Arrow permits arbitrary child values under a null parent, so masked nulls must not cause a false rejection. Visible nulls still fail at non-nullable levels. The name-only spatial fallback cannot preserve dimensions and CRS, so spatial metadata must be read fully. Visibility propagation currently covers structs and fixed-size lists. List views, maps, dictionaries, and run-end arrays remain follow-ups in #11114. Source checks passed. No builds or runtime tests were run for the split. ## API Changes - Known extension fields now fail on unsupported or malformed input instead of silently becoming storage arrays. - Nullable-marked vector elements and spatial coordinates, plus GeoArrow `LargeList` input, import as their extension when the visible values are valid. - `ArrowSession` rejects visible nulls in non-nullable children, including when the supplied schema is stricter than the Arrow array. - Adds `ArrowSession::from_arrow_storage`. --------- Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on #11513.
Round trips can pass when import and export make matching mistakes. Independent Arrow references are needed to check each direction against the intended extension contract.
Changes
Generators stay in their owning crates to avoid adding plugin dependencies to
vortex-arrow. Generated cases replace repetitive examples while retaining directed regressions and rejection tests. Native values, aggregates, and tensor layouts are checked separately so transport failures cannot hide failures in those contracts.Variant remains excluded pending #11359.
Source checks passed. No builds or runtime tests were run for the split.