Skip to content

[C++] Remove internal GroupBy implementation #14866

Closed

Description

@westonpace

Describe the enhancement requested

Currently there are two ways to compute a group by. The supported way is to use an aggregate node in an exec plan. The second (internal) way is to use the internal function arrow::internal::GroupBy. This internal function simulates, but does not actually use, an aggregate node.

The internal implementation has caused issues in the past where we did not notice an error in the aggregate node's invocation of aggregate kernels since we use the internal function for testing aggregates and it behaves slightly differently. The internal implementation also requires maintenance and significantly complicated #14352 .

I would like to remove the internal implementation. Unfortunately, the internal implementation is used by tests, benchmarks, and pyarrow. However, we should be able to update those bindings to a friendly wrapper around exec plans.

Component(s)

C++

Activity

  1. coady commented on Feb 11, 2023

    @coady

    I think the current _group_by interface is awkward, because it requires target "names", which are only used for function arity. But it looks like #14867 will replace that anyway.

    Also, I've been using _group_by for aggregating batches, with out-of-core data. Whereas #14867 looks like it will require a table. Just something to consider, but I guess I could always wrap a single record batch into a table.

    Thanks.

  2. westonpace commented on Feb 15, 2023

    @westonpace
    MemberAuthor

    Yes, wrapping a single batch in a table should be effective.

  3. added a commit that references this issue on Feb 17, 2023
  4. added a commit that references this issue on Feb 21, 2023
  5. added this to the 12.0.0 milestone on Feb 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions