You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently Scan generates an ordered sequence of batches. However, this is not ideal. For example, consider reading four parquet files in parallel from S3. There is no good way to determine which read will finish first. If file 2 finishes before file 1 then we could start parsing the contents of file 2 immediately but we currently do not.
There could then be an option provided by Scan whether to preserve ordering or not. Cases that do not care about ordering (e.g. count rows) could take advantage of this to reduce memory pressure.
Note: This will be an optimization even for cases that do care about ordering. We could still parse / project / etc. out of order and simply reorder at the end. The only difference between unordered and ordered then will be the memory pressure applied.
This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.
Activity
github-actions commentedon Jan 13, 2026
This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.