Skip to content

GH-47483: [C++] Bump vendored xxhash to 0.8.3 - #47476

Merged
pitrou merged 2 commits into
apache:mainfrom
raulcd:GH-47475
Sep 3, 2025
Merged

pitrou merged 2 commits into
apache:mainfrom
raulcd:GH-47475

Conversation

@raulcd

@raulcd raulcd commented Sep 2, 2025 •

Copy link
Copy Markdown
Member

Rationale for this change

We are using an older version, this might fix the current failure for test-conda-cpp-valgrind failures. The upgrade might also come with some improvements / fixes.

What changes are included in this PR?

Update vendored xxhash to newer version:

Are these changes tested?

Yes, via CI.

Are there any user-facing changes?

No

@github-actions

github-actions Bot commented Sep 2, 2025

Copy link
Copy Markdown

⚠️ GitHub issue #47475 has been automatically assigned in GitHub to PR creator.

@raulcd

raulcd commented Sep 2, 2025

Copy link
Copy Markdown
Member Author

Current failures are related to LLVM, I will rebase once the PR fixing those jobs is merged.

@raulcd

raulcd commented Sep 3, 2025

Copy link
Copy Markdown
Member Author

@github-actions crossbow submit -g cpp

@github-actions

github-actions Bot commented Sep 3, 2025

Copy link
Copy Markdown

Revision: 29e3ab6

Submitted crossbow builds: ursacomputing/crossbow @ actions-9f25a8e327

Task Status
example-cpp-minimal-build-static GitHub Actions
example-cpp-minimal-build-static-system-dependency GitHub Actions
example-cpp-tutorial GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1 GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-42-cpp GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-bundled GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-bundled-offline GitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundled GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizer GitHub Actions

@raulcd

raulcd commented Sep 3, 2025

Copy link
Copy Markdown
Member Author

It seems that the xxhash bump doesn't fix the test-conda-cpp-valgrind issues:
https://github.com/ursacomputing/crossbow/actions/runs/17428470869/job/49481058706

Still, seems worth updating it, @pitrou what do you think?
Should I create an issue for the version bump and we handle the job failure on the original issue?

@pitrou

pitrou commented Sep 3, 2025

Copy link
Copy Markdown
Member

Yes, it's worth bumping xxhash anyway.

@raulcd raulcd changed the title GH-47475: [C++] Bump vendored xxhash to 0.8.3 GH-47483: [C++] Bump vendored xxhash to 0.8.3 Sep 3, 2025
@github-actions

github-actions Bot commented Sep 3, 2025

Copy link
Copy Markdown

⚠️ GitHub issue #47483 has been automatically assigned in GitHub to PR creator.

@raulcd
raulcd marked this pull request as ready for review September 3, 2025 11:08
@raulcd
raulcd requested a review from pitrou September 3, 2025 11:11
@pitrou

pitrou commented Sep 3, 2025

Copy link
Copy Markdown
Member

It appears this was reported and discussed upstream: Cyan4973/xxHash#943 (comment)

We should probably apply the stated workaround:

the only thing I can suggest at this point is to recommend using XXH_NO_INLINE_HINTS when compiling with -Og

@raulcd

raulcd commented Sep 3, 2025

Copy link
Copy Markdown
Member Author

@github-actions crossbow submit test-conda-cpp-valgrind

@github-actions

github-actions Bot commented Sep 3, 2025

Copy link
Copy Markdown

Revision: eb76ca8

Submitted crossbow builds: ursacomputing/crossbow @ actions-d2be094bc2

Task Status
test-conda-cpp-valgrind GitHub Actions

Comment thread cpp/CMakeLists.txt

include(SetupCxxFlags)

if(${CMAKE_CXX_FLAGS_DEBUG} MATCHES "-Og")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kou Does it look like the right way to do this?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No.

In general, we should not use add_definitions() because its scope is global. We should create a target and use it instead:

add_library(arrow::xxhash INTERFACE IMPORTED)
target_compile_definitions(arrow::xxhash INTERFACE "$<$<CONFIG:Debug>:XXH_NO_INLINE_HINTS>")

(I think that we don't need -Og check. We can always disable inline hints in debug build.)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: #47495 is an approach that doesn't use add_definitions().

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou , should we create targets for the different vendored libraries and only link when necessary as a future task too? I am thinking about datetime, double-conversion, etc

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes when they need additional build flags. If they don't need additional build flags, we don't need to create targets for them. Because we have src/ in include paths:

arrow/cpp/CMakeLists.txt

Lines 559 to 560 in 743d0af

include_directories(${CMAKE_CURRENT_BINARY_DIR}/src)
include_directories(src)

BTW, we should not use include_directories() because its scope is global (directory) too...

@raulcd

raulcd commented Sep 3, 2025

Copy link
Copy Markdown
Member Author

The CI failure is now unrelated to building but to a leak on azure-test.

The following tests FAILED:
	 77 - arrow-azurefs-test (Failed)

@pitrou

pitrou commented Sep 3, 2025

Copy link
Copy Markdown
Member

Cool, I suspect we can merge but let's just first wait for AppVeyor to finish?

@pitrou

pitrou commented Sep 3, 2025

Copy link
Copy Markdown
Member

I restarted the Valgrind job. If the Azure-related failure occurs again, we'll have to file a separate issue for it.

@raulcd

raulcd commented Sep 3, 2025

Copy link
Copy Markdown
Member Author

I restarted the Valgrind job. If the Azure-related failure occurs again, we'll have to file a separate issue for it.

It has failed again, I've opened a new issue to track the Azure leak:

@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 2284234.

There weren't enough matching historic benchmark results to make a call on whether there were regressions.

The full Conbench report has more details.

Mottl pushed a commit to Mottl/arrow that referenced this pull request May 26, 2026
### Rationale for this change

We are using an older version, this might fix the current failure for test-conda-cpp-valgrind failures. The upgrade might also come with some improvements / fixes.

### What changes are included in this PR?

Update vendored xxhash to newer version:
- https://github.com/Cyan4973/xxHash/releases/tag/v0.8.3

### Are these changes tested?

Yes, via CI.

### Are there any user-facing changes?

No

* GitHub Issue: apache#47483

Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants