Skip to content

GH-44579: [C++] Use array type to compute min/max statistics Arrow type - #45094

Merged
kou merged 1 commit into
apache:mainfrom
kou:cpp-array-statistics-binary
Dec 31, 2024
Merged

kou merged 1 commit into
apache:mainfrom
kou:cpp-array-statistics-binary

Conversation

@kou

@kou kou commented Dec 21, 2024 •

Copy link
Copy Markdown
Member

Rationale for this change

arrow::ArrayStatistics uses raw C++ types such as int64_t and std::string for min/max types. We need to convert raw C++ types to Arrow types when we use arrow::ArrayStatistics for generating statistics array. (GH-45038)

We can't map std::string to an Arrow type. Because it may be arrow::binary(), arrow::utf8() or something.

What changes are included in this PR?

Use arrow::DataType information of associated array when we convert arrow::ArrayStatistics's min/max raw C++ types to Arrow types.

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #44579 has been automatically assigned in GitHub to PR creator.

switch (array_type->id()) {
case Type::STRING:
case Type::BINARY:
case Type::FIXED_SIZE_BINARY:

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use arrow::binary() or arrow::large_binary() for Type::FIXED_SIZE_BINARY?
Approximate min/max values may not be a fixed size binary...?

@kou

kou commented Dec 25, 2024

Copy link
Copy Markdown
Member Author

I'll merge this in the next week if nobody objects this.

@kou
kou merged commit fd1bf8e into apache:main Dec 31, 2024
@kou
kou deleted the cpp-array-statistics-binary branch December 31, 2024 03:05
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit fd1bf8e.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 10 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant