Skip to content

[C++] Add unordered scan #27651

Closed as not planned
Closed as not planned

Description

@asfimport

Currently Scan generates an ordered sequence of batches.  However, this is not ideal.  For example, consider reading four parquet files in parallel from S3.  There is no good way to determine which read will finish first.  If file 2 finishes before file 1 then we could start parsing the contents of file 2 immediately but we currently do not.

There could then be an option provided by Scan whether to preserve ordering or not.  Cases that do not care about ordering (e.g. count rows) could take advantage of this to reduce memory pressure.

Note: This will be an optimization even for cases that do care about ordering.  We could still parse / project / etc. out of order and simply reorder at the end.  The only difference between unordered and ordered then will be the memory pressure applied.

Reporter: Weston Pace / @westonpace

Note: This issue was originally created as ARROW-11800. Please see the migration documentation for further details.

Activity

  1. github-actions commented on Jan 13, 2026

    @github-actions

    This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.

  2. added
    Status: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwise
    on Jan 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions