Skip to content

[Python] Add missing APIs to RecordBatch class #36399

Open

Description

@danepitkin

Describe the enhancement requested

The following APIs exist in the Table class, but not in the RecordBatch class:

add_column
append_column
cast
drop
drop_columns
group_by
join
rename_columns
remove_columns
set_column
shape

Note that the following Table APIs are not applicable to RecordBatch.
combine_chunks
flatten
from_batches
to_batches
to_reader

Component(s)

Python

Activity

  1. jorisvandenbossche commented on Jun 30, 2023

    @jorisvandenbossche
    Member

    Thanks for the overview!

    I would personally start with the methods to manipulate the RecordBatch, i.e. the ones for adding/removing/renaming columns, and cast.

  2. jorisvandenbossche commented on Mar 15, 2024

    @jorisvandenbossche
    Member

    With #39506 merged, the remaining relevant methods missing on RecordBatch are:

    • group_by
    • join
    • shape
  3. added a commit that references this issue on Mar 15, 2024
  4. judahrand commented on Mar 18, 2024

    @judahrand
    Contributor

    With #39506 merged, the remaining relevant methods missing on RecordBatch are:

    • group_by
    • join
    • shape

    And now that #34234 is merged join_asof as well?

  5. judahrand commented on Mar 18, 2024

    @judahrand
    Contributor

    I think there is also an open question as to what type join and join_asof should return. I'm not sure it makes sense for these methods to return a RecordBatch since, depending on the data and join type, they may return many more rows than either of the input RecordBatches. The obvious option would then be to return a Table or a RecordBatchReader. A Table might be more immediately useful to consumers as it allows for methods to be chained without calling .read_all, however, a RecordBatchReader in some ways makes more sense given that cases where the users is operating on a RecordBatch are likely to be cases where streaming type workflows are preferable.

    What do you think @jorisvandenbossche?

  6. added a commit that references this issue on Mar 21, 2024
  7. added
    Status: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwise
    on Feb 18, 2026
  8. thisisnic commented on Feb 18, 2026

    @thisisnic
    Member

    This issue has been marked as stale because it has had no activity in the past 2 years. Please remove the stale label or comment below, or this issue will be closed in 14 days.

  9. removed
    Status: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwise
    on Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions