Skip to content

[C++][Parquet] Parquet arrow::FileWriter accepts arrow::RecordBatch in the buffered row group mode #15216

Closed

Description

@wgtmac

Describe the enhancement requested

Current state of the parquet arrow::FileWriter

  • It does not support buffered row group mode. User has to write the whole column chunk one by one.
  • It only accepts arrow::Table in the tabular form, or arrow::Array/arrow::ChunkedArray in the columnar form. If one only has arrow::RecordBatch on hand, additional transformation is required before calling the writer.

The goal of this issue aims to enhance the arrow::FileWriter as below:

  • Accept writing arrow::RecordBatch.
  • Support buffered row group mode.

Component(s)

C++, Parquet

Activity

  1. added a commit that references this issue on Jan 12, 2023
  2. added a commit that references this issue on Jan 12, 2023
  3. added this to the 11.0.0 milestone on Jan 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions