Skip to content

[C++] Allow ParquetWriter to take a RecordBatchReader as input #30279

Closed as not planned

Description

@asfimport

Please could we extend the Parquet Writer to not only take a Table or RecordBatch as inputs, but also RecordBatchReader? This would be super-helpful for opening data as a dataset and writing to a single file.

Reporter: Nicola Crane / @thisisnic

Related issues:

PRs and other links:

Note: This issue was originally created as ARROW-14742. Please see the migration documentation for further details.

Activity

  1. asfimport commented on Aug 18, 2022

    @asfimport
    Author

    Alvin Chunga Mamani / @AlvinJ15:
    The current implementation of the ParquetFileWriter  inherits the 

    FileWriter::Write(RecordBatchReader* batches)

    {{{}}}which is  the RecordBatchReader support.
    This is Similar as CSVFileWriter
    cc:   @thisisnic  

  2. asfimport commented on Aug 30, 2022

    @asfimport
    Author

    Nicola Crane / @thisisnic:
    I'll admit I don't understand this bit of the codebase super well, and so will have to take a look later to see if this means that ARROW-14428 is currently possible.

  3. asfimport commented on Nov 28, 2022

    @asfimport
    Author

    Apache Arrow JIRA Bot:
    This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned per project policy. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.

  4. wjones127 commented on Jan 11, 2023

    @wjones127
    Member

    Once #15216 is complete, this should be easy to implement without requiring Dataset. The method @AlvinJ15 mentions is from the dataset module.

  5. github-actions commented on Feb 25, 2026

    @github-actions

    This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.

  6. added
    Status: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwise
    on Feb 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions