Skip to content

GH-45589: [C++] Enable singular test in Meson configuration - #45596

Merged
WillAyd merged 6 commits into
apache:mainfrom
WillAyd:meson-test
Mar 12, 2025
Merged

WillAyd merged 6 commits into
apache:mainfrom
WillAyd:meson-test

Conversation

@WillAyd

@WillAyd WillAyd commented Feb 20, 2025 •

Copy link
Copy Markdown
Contributor

Rationale for this change

Adding this singular test helps us make incremental progress on Meson as a build system generator

What changes are included in this PR?

A single array-test for libarrow is being added

Are these changes tested?

Yes - see CI

Are there any user-facing changes?

No - this is only for developers

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #45589 has been automatically assigned in GitHub to PR creator.

Comment thread cpp/src/arrow/meson.build Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The static_library is required here because the Meson wrap dependency for gtest only provides consumers with the source files, and when using that in two different shared libraries then linking together you get an ODR violation.

See also mesonbuild/wrapdb#1937

Comment thread cpp/meson.build Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to follow along with the CMake configuration. The option doesn't yet exist in Meson so I've hard-coded for now

Comment thread ci/scripts/cpp_build.sh Outdated
Comment on lines 113 to 116

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
meson setup \
--prefix=${MESON_PREFIX:-${ARROW_HOME}} \
--buildtype=${ARROW_BUILD_TYPE:-debug} \
-Dtests=true \
function meson_boolean() {
if [ "${1}" = "ON" ]; then
echo "true"
else
echo "false"
fi
}
meson setup \
--prefix=${MESON_PREFIX:-${ARROW_HOME}} \
--buildtype=${ARROW_BUILD_TYPE:-debug} \
-Dtests=$(meson_boolean ${ARROW_BUILD_TESTS:-OFF}) \

Comment thread cpp/.gitignore Outdated
Comment on lines 49 to 51

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
subprojects/*
!subprojects/packagefiles
!subprojects/*.wrap
/subprojects/*
!/subprojects/packagefiles
!/subprojects/*.wrap

@kou

kou commented Feb 22, 2025

Copy link
Copy Markdown
Member

@github-actions crossbow submit test-conda-cpp-meson

@github-actions

Copy link
Copy Markdown

Revision: 65779fa641362cc3e960133fa51e7ce928afa620

Submitted crossbow builds: ursacomputing/crossbow @ actions-b0685743f1

Task Status
test-conda-cpp-meson GitHub Actions

@kou

kou commented Feb 22, 2025

Copy link
Copy Markdown
Member

https://github.com/ursacomputing/crossbow/actions/runs/13468923214/job/37639840316#step:6:448

Run-time dependency rapidjson found: NO (tried pkgconfig and cmake)

Hmm. It's strange that RapidJSON wasn't found. RapidJSON exists in the environment:

https://github.com/ursacomputing/crossbow/actions/runs/13468923214/job/37639840316#step:6:351

rapidjson                 1.1.0.post20240409      hac33072_1    conda-forge

@WillAyd

WillAyd commented Feb 25, 2025

Copy link
Copy Markdown
Contributor Author

Sorry I've been away for a few days.

Hmm. It's strange that RapidJSON wasn't found. RapidJSON exists in the environment:

In spite of Meson not finding the system RapidJSON, it does still fall back to using the wrap file to download an appropriate copy. Seems like the error message from that run has to do with resolving the filepath to the bundled flatbuffers. Will investigate further

@WillAyd

WillAyd commented Feb 25, 2025 •

Copy link
Copy Markdown
Contributor Author

FWIW it might even be worthwhile to use Meson's wrap and subproject systems for flatbuffers instead of using a vendored copy. The limiting factor was that Arrow seems to statically assert the required major/minor version of flatbuffers, and the minor version requested does not exist in the wrapdb.

If there is some flexibility on that version check we could go down that path

@kou

kou commented Feb 26, 2025

Copy link
Copy Markdown
Member

We should add support for system Flatbuffers, right? (We can fallback to the vendored Flatbuffers and generated files when system Flatbuffers don't exist.)

Could you open an issue for this? Let's discuss this in the new issue.

@WillAyd

WillAyd commented Mar 10, 2025

Copy link
Copy Markdown
Contributor Author

@github-actions crossbow submit test-conda-cpp-meson

@github-actions

Copy link
Copy Markdown

Revision: c438216

Submitted crossbow builds: ursacomputing/crossbow @ actions-04885d5223

Task Status
test-conda-cpp-meson GitHub Actions

@WillAyd

WillAyd commented Mar 10, 2025

Copy link
Copy Markdown
Contributor Author

Looks like the current failure is due to a lack of boost in the crossbow environment. Boost doesn't exist in the wrapdb given its current size, but I might be able to create some custom wraps for it. Need to investigate further

@WillAyd

WillAyd commented Mar 10, 2025

Copy link
Copy Markdown
Contributor Author

@github-actions crossbow submit test-conda-cpp-meson

@github-actions

Copy link
Copy Markdown

Revision: 3ea91f2699ec56e327d1398c1eb75133d196130d

Submitted crossbow builds: ursacomputing/crossbow @ actions-c1928ec294

Task Status
test-conda-cpp-meson GitHub Actions

@WillAyd
WillAyd force-pushed the meson-test branch 2 times, most recently from 758a88b to f000595 Compare March 10, 2025 23:30
Comment thread cpp/subprojects/boost.wrap Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes the initial setup pretty slow because it recurses a lot of unnecessary submodules in the boost repo. Didn't want to try and solve that now, but would be good to eventually fix

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use boost-X.Y.Z-cmake.tar.xz instead?
For example, boost-1.87.0-cmake.tar.xz exists at https://github.com/boostorg/boost/releases/tag/boost-1.87.0 .

@WillAyd

WillAyd commented Mar 10, 2025

Copy link
Copy Markdown
Contributor Author

@github-actions crossbow submit test-conda-cpp-meson

@github-actions

Copy link
Copy Markdown

Revision: f0005959d735d8e4aeba3b478d8d5b6db45551e6

Submitted crossbow builds: ursacomputing/crossbow @ actions-0c64ef7f5a

Task Status
test-conda-cpp-meson GitHub Actions

Comment thread cpp/src/arrow/meson.build Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need system too?

Suggested change
modules: ['filesystem'],
modules: ['filesystem', 'system'],

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe we need that at the moment. If we do, I think we might need to also have filesystem_dep and system_dep be separate dependencies rather than included in one (unless we don't ever care to differentiate between them)

Comment thread cpp/subprojects/boost.wrap Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use boost-X.Y.Z-cmake.tar.xz instead?
For example, boost-1.87.0-cmake.tar.xz exists at https://github.com/boostorg/boost/releases/tag/boost-1.87.0 .

@WillAyd

WillAyd commented Mar 11, 2025

Copy link
Copy Markdown
Contributor Author

@github-actions crossbow submit test-conda-cpp-meson

@github-actions

Copy link
Copy Markdown

Revision: b4b615f

Submitted crossbow builds: ursacomputing/crossbow @ actions-122520e76a

Task Status
test-conda-cpp-meson GitHub Actions

@kou kou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Comment thread cpp/subprojects/boost.wrap Outdated
source_filename = boost-1.87.0-cmake.tar.gz
source_hash = 78fbf579e3caf0f47517d3fb4d9301852c3154bfecdc5eeebd9b2b0292366f5b
directory = boost-1.87.0
method=cmake

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
method=cmake
method = cmake

@WillAyd
WillAyd merged commit b9d014d into apache:main Mar 12, 2025
@WillAyd
WillAyd deleted the meson-test branch March 12, 2025 14:46
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b9d014d.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 6 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants