Repository navigation
Add a layout for struct-backed extensions - #11640
connortsui20 wants to merge 1 commit into
Conversation
36f0650 to
026f828
Compare
A struct-backed extension written as a leaf makes a field query load every storage field. Let types opt into a structural layout so explicit storage reads can use the existing struct reader and planner without losing the extension dtype. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
026f828 to
24fd9da
Compare
Merging this PR will degrade performance by 14.56%2 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
The branch carried the prototype versions of the changes that the open PRs make, and those PRs have moved on in review. For example, #11640 now names the layout `vortex.extension`. This merge takes the tree of a stack on develop at 967bcb6, which holds the PR heads and ports of the two changes that have no PR yet: - #11640 at 24fd9da. - #11710 at add4115. - #11639 at 4f25e2f. Its newer head, 115318b, fails three of its own tests. - #11642 at d16038c. - Zone maps of bounds beneath nullable structs. - Single fields of list elements on both scans. A merge instead of a force-push keeps every commit that the branch published reachable. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
This PR adds
vortex.extension, an experimental layout for struct-backed extension types. Ondevelop, the file writer stores these types in a leaf layout, so a scan reads the whole value even when it needs only one storage field. The new layout keeps the extension dtype and wraps a struct layout that stores the fields separately.For a
Pointbacked by{x, y},get_item("x", ext_storage(root()))can readxwithout readingy, and a filter onxcan use its zone map. A nullablePointgets field zone maps once #11641 lands, as any nullable struct does. A whole-value read still returns aPoint.The default writer requires the unstable edition and per-type opt-in. Splitting the storage removes whole-value zone maps, so each type chooses whether to use the new layout. Prototype design.
ForeignExtDTypeis public under the_test-harnessfeature, so that the layout and file tests can build extension dtypes over any storage dtype without a vtable for each shape.This change was implemented and reviewed with OpenAI Codex and Claude Code.