Repository navigation
Conversation
VortexFile::plan_data_source scans one file through PlanDataSource, with the file's statistics pruning scans they rule out, and MultiFileDataSource::build_plan scans many, opening each file after the first as a scan reaches it. Both are opt-in: data_source() and build() still scan with layout readers. build_plan falls back to build() when the schema could hold a list layout, which the V2 scan cannot read yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Exposes Vortex files through
PlanDataSource:VortexFile::plan_data_sourcescans one file.VortexFile::plan_childmakes a file a child of any plan source.MultiFileDataSource::build_planscans many files. It opens the first file for the schema and every later one only when a scan reaches it.layout_reader_data_sourcenames the layout-reader path explicitly.data_source()andbuild()keep scanning on layout readers, so this PR changes no default.Contracts to check:
ScanCache: the file's own withwith_caching, otherwise one owned by the source.build_planfalls back tobuild's layout-reader source whenever the schema could hold one.may_contain_list_layoutis conservative: maps and variants count.Stacked on #11699.
🤖 Generated with Claude Code