Skip to content

[Benchmarking][CI] Benchmarking buildkite runs fail to build Arrow #45939

Closed

Description

@raulcd

Describe the bug, including details regarding any error messages, version, and platform.

As noted by @zanmato1984 on this PR:

Benchmarks have stopped running. The latest benchmarked commit PR merged is:
#45929
The next one merged doesn't seem to have benchmarks run:
#45907
Which matches with the runs here:
https://conbench.ursa.dev/
It does seem that buildkite runs are failing with the following error:

[445/640] Building CXX object src/arrow/c/CMakeFiles/arrow-bridge-benchmark.dir/bridge_benchmark.cc.o
FAILED: src/arrow/c/CMakeFiles/arrow-bridge-benchmark.dir/bridge_benchmark.cc.o
/var/lib/buildkite-agent/.conda/envs/arrow-commit/bin/ccache /var/lib/buildkite-agent/.conda/envs/arrow-commit/bin/x86_64-conda-linux-gnu-c++ -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_AVX512 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_WITH_BENCHMARKS_REFERENCE -DARROW_WITH_TIMING_TESTS -I/tmp/5909f767-7a62-48fc-a33c-79508cc22d8b/cpp/src -I/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src -I/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/generated -isystem /var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/thirdparty/flatbuffers/include -isystem /tmp/5909f767-7a62-48fc-a33c-79508cc22d8b/cpp/_deps/googletest-src/googletest/include -isystem /tmp/5909f767-7a62-48fc-a33c-79508cc22d8b/cpp/_deps/googletest-src/googletest -isystem /tmp/5909f767-7a62-48fc-a33c-79508cc22d8b/cpp/_deps/googletest-src/googlemock/include -isystem /tmp/5909f767-7a62-48fc-a33c-79508cc22d8b/cpp/_deps/googletest-src/googlemock -Wredundant-move -Wno-noexcept-type -Wno-self-move -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /var/lib/buildkite-agent/.conda/envs/arrow-commit/include -fdiagnostics-color=always  -Wall -Wno-conversion -Wno-sign-conversion -Wdate-time -Wimplicit-fallthrough -Wunused-result -fno-semantic-interposition -msse4.2  -O3 -DNDEBUG -O2 -ftree-vectorize  -std=c++17 -fPIE -pthread -MD -MT src/arrow/c/CMakeFiles/arrow-bridge-benchmark.dir/bridge_benchmark.cc.o -MF src/arrow/c/CMakeFiles/arrow-bridge-benchmark.dir/bridge_benchmark.cc.o.d -o src/arrow/c/CMakeFiles/arrow-bridge-benchmark.dir/bridge_benchmark.cc.o -c /var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc
In file included from /var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:20:
/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:149:1: error: no matching function for call to 'make_unique<benchmark::internal::FunctionBenchmark>(const char [11], <unresolved overloaded function type>)'
  149 | BENCHMARK(ExportType);
      | ^~~~~~~~~
In file included from /var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/memory:78,
                 from /var/lib/buildkite-agent/.conda/envs/arrow-commit/include/benchmark/benchmark.h:177:
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1094:5: note: candidate: 'template<class _Tp, class ... _Args> std::__detail::__invalid_make_unique_t<_Tp> std::make_unique(_Args&& ...)' (deleted)
 1094 |     make_unique(_Args&&...) = delete;
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1094:5: note:   template argument deduction/substitution failed:
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h: In substitution of 'template<class _Tp> using std::__detail::__invalid_make_unique_t = typename std::__detail::_MakeUniq::__invalid_type [with _Tp = benchmark::internal::FunctionBenchmark]':
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1094:5:   required by substitution of 'template<class _Tp, class ... _Args> std::__detail::__invalid_make_unique_t<_Tp> std::make_unique(_Args&& ...) [with _Tp = benchmark::internal::FunctionBenchmark; _Args = <missing>]'
/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:149:1:   required from here
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1055:11: error: no type named '__invalid_type' in 'struct std::__detail::_MakeUniq<benchmark::internal::FunctionBenchmark>'
 1055 |     using __invalid_make_unique_t = typename _MakeUniq<_Tp>::__invalid_type;
      |           ^~~~~~~~~~~~~~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1084:5: note: candidate: 'template<class _Tp> std::__detail::__unique_ptr_array_t<_Tp> std::make_unique(size_t)'
 1084 |     make_unique(size_t __num)
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1084:5: note:   candidate expects 1 argument, 2 provided
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1069:5: note: candidate: 'std::__detail::__unique_ptr_t<_Tp> std::make_unique(_Args&& ...) [with _Tp = benchmark::internal::FunctionBenchmark; _Args = {}; __detail::__unique_ptr_t<_Tp> = __detail::__unique_ptr_t<benchmark::internal::FunctionBenchmark>]'
 1069 |     make_unique(_Args&&... __args)
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1069:5: note:   candidate expects 0 arguments, 2 provided
/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:150:1: error: no matching function for call to 'make_unique<benchmark::internal::FunctionBenchmark>(const char [13], <unresolved overloaded function type>)'
  150 | BENCHMARK(ExportSchema);
      | ^~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1094:5: note: candidate: 'template<class _Tp, class ... _Args> std::__detail::__invalid_make_unique_t<_Tp> std::make_unique(_Args&& ...)' (deleted)
 1094 |     make_unique(_Args&&...) = delete;
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1094:5: note:   template argument deduction/substitution failed:
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1084:5: note: candidate: 'template<class _Tp> std::__detail::__unique_ptr_array_t<_Tp> std::make_unique(size_t)'
 1084 |     make_unique(size_t __num)
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1084:5: note:   candidate expects 1 argument, 2 provided
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1069:5: note: candidate: 'std::__detail::__unique_ptr_t<_Tp> std::make_unique(_Args&& ...) [with _Tp = benchmark::internal::FunctionBenchmark; _Args = {}; __detail::__unique_ptr_t<_Tp> = __detail::__unique_ptr_t<benchmark::internal::FunctionBenchmark>]'
 1069 |     make_unique(_Args&&... __args)
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1069:5: note:   candidate expects 0 arguments, 2 provided
/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:151:1: error: no matching function for call to 'make_unique<benchmark::internal::FunctionBenchmark>(const char [12], <unresolved overloaded function type>)'
  151 | BENCHMARK(ExportArray);
      | ^~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1094:5: note: candidate: 'template<class _Tp, class ... _Args> std::__detail::__invalid_make_unique_t<_Tp> std::make_unique(_Args&& ...)' (deleted)
 1094 |     make_unique(_Args&&...) = delete;
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1094:5: note:   template argument deduction/substitution failed:
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1084:5: note: candidate: 'template<class _Tp> std::__detail::__unique_ptr_array_t<_Tp> std::make_unique(size_t)'
 1084 |     make_unique(size_t __num)
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1084:5: note:   candidate expects 1 argument, 2 provided
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1069:5: note: candidate: 'std::__detail::__unique_ptr_t<_Tp> std::make_unique(_Args&& ...) [with _Tp = benchmark::internal::FunctionBenchmark; _Args = {}; __detail::__unique_ptr_t<_Tp> = __detail::__unique_ptr_t<benchmark::internal::FunctionBenchmark>]'
 1069 |     make_unique(_Args&&... __args)
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1069:5: note:   candidate expects 0 arguments, 2 provided
/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:152:1: error: no matching function for call to 'make_unique<benchmark::internal::FunctionBenchmark>(const char [18], <unresolved overloaded function type>)'
  152 | BENCHMARK(ExportRecordBatch);
      | ^~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1094:5: note: candidate: 'template<class _Tp, class ... _Args> std::__detail::__invalid_make_unique_t<_Tp> std::make_unique(_Args&& ...)' (deleted)
 1094 |     make_unique(_Args&&...) = delete;
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1094:5: note:   template argument deduction/substitution failed:
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1084:5: note: candidate: 'template<class _Tp> std::__detail::__unique_ptr_array_t<_Tp> std::make_unique(size_t)'
 1084 |     make_unique(size_t __num)
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1084:5: note:   candidate expects 1 argument, 2 provided
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1069:5: note: candidate: 'std::__detail::__unique_ptr_t<_Tp> std::make_unique(_Args&& ...) [with _Tp = benchmark::internal::FunctionBenchmark; _Args = {}; __detail::__unique_ptr_t<_Tp> = __detail::__unique_ptr_t<benchmark::internal::FunctionBenchmark>]'
 1069 |     make_unique(_Args&&... __args)
      |     ^~~~~~~~~~~
/var/lib/buildkite-agent/.conda/envs/arrow-commit/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1069:5: note:   candidate expects 0 arguments, 2 provided
/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:91:13: warning: 'void arrow::ExportRecordBatch(benchmark::State&)' defined but not used [-Wunused-function]
   91 | static void ExportRecordBatch(benchmark::State& state) {  // NOLINT non-const reference
      |             ^~~~~~~~~~~~~~~~~
/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:80:13: warning: 'void arrow::ExportArray(benchmark::State&)' defined but not used [-Wunused-function]
   80 | static void ExportArray(benchmark::State& state) {  // NOLINT non-const reference
      |             ^~~~~~~~~~~
/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:69:13: warning: 'void arrow::ExportSchema(benchmark::State&)' defined but not used [-Wunused-function]
   69 | static void ExportSchema(benchmark::State& state) {  // NOLINT non-const reference
      |             ^~~~~~~~~~~~
/var/lib/buildkite-agent/builds/amd64-c6a-4xlarge-linux-i-0e8310a07745abccb-1/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/arrow/cpp/src/arrow/c/bridge_benchmark.cc:58:13: warning: 'void arrow::ExportType(benchmark::State&)' defined but not used [-Wunused-function]
   58 | static void ExportType(benchmark::State& state) {  // NOLINT non-const reference
      |             ^~~~~~~~~~
[461/640] Building CXX object src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/scanner.cc.o
ninja: build stopped: subcommand failed.

Component(s)

Benchmarking

Activity

  1. added this to the 20.0.0 milestone on Mar 26, 2025
  2. raulcd commented on Mar 26, 2025

    @raulcd
    MemberAuthor

    There was a new release of Google Benchmark yesterday (1.9.2) which is the one used on the run and checks with the problem:
    https://github.com/google/benchmark/releases/tag/v1.9.2

  3. raulcd commented on Mar 26, 2025

    @raulcd
    MemberAuthor

    It does seem related to this change:

    But I am unsure on the fix, @pitrou any idea?

  4. pitrou commented on Mar 26, 2025

    @pitrou
    Member

    I think this might be because the benchmark function names (ExportType, etc.) clash with the public API functions they benchmark, so std::make_unique is not able to resolve a single signature. I'll try a fix.

  5. pitrou commented on Mar 26, 2025

    @pitrou
    Member

    Issue resolved by pull request 45942
    #45942

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions