Repository navigation
fix(python): VortexDataset.take accepts indices in any order - #11410
Merged
robert3005 merged 2 commits intoNov 16, 2026
Merged
Conversation
The scan layer requires strictly increasing row indices, so take([7, 2]) and take([7, 2, 7]) failed with "buffer values must be strictly increasing" while pyarrow's Dataset.take returns the rows in the order requested. When the indices are not strictly increasing and no row filter applies, scan the sorted distinct rows once and gather them back into the requested order. Sorted input keeps the existing single-scan path. With a filter, from the argument or from Dataset.filter, the surviving rows cannot be mapped back to positions, so the indices must still be strictly increasing; the docstring says so. Signed-off-by: jackylee-ch <qcsd2011@gmail.com>
robert3005
approved these changes
Nov 12, 2026
robert3005
enabled auto-merge (squash)
November 12, 2026 12:36
Annotate the take helpers with pyarrow.Array[Any] and suppress two pyarrow-stubs overload mismatches so ty 0.0.98 passes. Signed-off-by: jackylee-ch <qcsd2011@gmail.com>
auto-merge was automatically disabled
November 12, 2026 13:12
Head branch was pushed to by a user without write access
robert3005
enabled auto-merge (squash)
November 12, 2026 13:35
robert3005
approved these changes
Nov 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
VortexDataset.takefailed with "buffer values must be strictly increasing" fortake([7, 2])andtake([7, 2, 7]), while pyarrow'sDataset.takereturns the rows in the requested order.Changes
Array.take. Sorted input keeps the single-scan path.filterargument orDataset.filter) the surviving rows cannot be mapped back to positions, so strictly increasing indices are still required. The docstring says so.AI assistance
Written with agentic AI assistance.