Skip to content

[C++] Compilation error when linking arrow-flight-perf-server #24752

Closed
Closed
[C++] Compilation error when linking arrow-flight-perf-server#24752
Assignees
pitrou
Labels
Component: BenchmarkingComponent: C++Component: FlightRPCType: bug
Milestone
1.0.0

Description

@asfimport
asfimport
opened on Apr 24, 2020

I wanted to play around with Flight benchmark after seeing the discussion regarding Flight's throughput in arrow dev mailing list today.

I met the following error when trying to build the benchmark from latest source code:

[ 95%] Linking CXX executable ../../../debug/arrow-flight-perf-server
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::filesystem::detail::canonical(boost::filesystem::path const&, boost::filesystem::path const&, boost::system::error_code*)'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::system::system_category()'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::filesystem::path::parent_path() const'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `deflate'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `deflateEnd'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::system::generic_category()'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::filesystem::detail::current_path(boost::system::error_code*)'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `inflateInit2_'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `inflate'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `deflateInit2_'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `inflateEnd'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::filesystem::path::operator/=(boost::filesystem::path const&)'
collect2: error: ld returned 1 exit status
src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/build.make:154: recipe for target 'debug/arrow-flight-perf-server' failed
make[2]: *** [debug/arrow-flight-perf-server] Error 1
CMakeFiles/Makefile2:2609: recipe for target 'src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all' failed
make[1]: *** [src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

I was using cmake .. -DCMAKE_BUILD_TYPE=Debug -DARROW_DEPENDENCY_SOURCE=AUTO -DARROW_FLIGHT=ON -DARROW_BUILD_BENCHMARKS=ON -DARROW_CXXFLAGS="-lboost_filesystem -lboost_system" to configure the build.
I noticed that there was a ARROW_BOOST_BUILD_VERSION: 1.71.0 in the output, but the Boost library that I installed from the package manger was of this version: 1.65.1.0ubuntu1. Could this be the cause of the problem?

PS:
I was able to build the benchmark before. It was on AWS with the OS being ubuntu-bionic-18.04-amd64-server-20191002, which should be very similar to the one I'm using on my laptop.

Environment: Linux HP 5.3.0-46-generic #38~18.04.1-Ubuntu SMP Tue Mar 31 04:17:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Reporter: Chengxin Ma / @MaChengxin
Assignee: Antoine Pitrou / @pitrou

PRs and other links:

Note: This issue was originally created as ARROW-8587. Please see the migration documentation for further details.

Activity

  1. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Antoine Pitrou / @pitrou:
    Weirdly, I get the same error now on Ubuntu 18.04. I used to be able to build it, so something broke along the way.

  2. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Chengxin Ma / @MaChengxin:
    Additional information: I still saw this error after rolling back the code base by: git checkout apache-arrow-0.17.0

  3. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Antoine Pitrou / @pitrou:
    I've bisected and the culprit is ARROW-7869.

  4. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Antoine Pitrou / @pitrou:
    By the way, you should build the benchmarks in release mode, not debug.

  5. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Wes McKinney / @wesm:
    Issue resolved by pull request 7031
    #7031

  6. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Chengxin Ma / @MaChengxin:
    Thanks for the quick fix.

    Unfortunately I still see the following error messages:

    [ 96%] Linking CXX executable ../../../release/arrow-flight-perf-server
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflateInit2_'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflate'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflateInit2_'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflate'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflateEnd'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflateEnd'
collect2: error: ld returned 1 exit status
src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/build.make:156: recipe for target 'release/arrow-flight-perf-server' failed
make[2]: *** [release/arrow-flight-perf-server] Error 1
CMakeFiles/Makefile2:2648: recipe for target 'src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all' failed
make[1]: *** [src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

    This seems to be a problem related to zlib. On my computer it is the latest version: zlib1g-dev is already the newest version (1:1.2.11.dfsg-0ubuntu2).

    I guess this issue is still related to ThirdpartyToolchain.cmake?

  7. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Antoine Pitrou / @pitrou:
    I don't see that error myself. Can you try to pass -DARROW_WITH_ZSTD=on ?

  8. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Chengxin Ma / @MaChengxin:
    Adding -DARROW_WITH_ZLIB=ON solved this problem.

    (I was expecting that the build system could find zlib on my system automatically so I didn't set this flag.)

  9. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Wes McKinney / @wesm:
    Is the zlib dependency coming from gRPC? It shouldn't be necessary to add ARROW_WITH_ZLIB=ON here (so this should be enabled automatically if it's needed)

  10. asfimport commented on Apr 24, 2020

    @asfimport
    asfimport
    on Apr 24, 2020
    Author

    Antoine Pitrou / @pitrou:
    I suppose so. I had no idea that gRPC required zlib (probably for optional compression, though we don't use it).

  11. asfimport commented on Apr 25, 2020

    @asfimport
    asfimport
    on Apr 25, 2020
    Author

    Chengxin Ma / @MaChengxin:
    I've built the Flight benchmark on different platforms before and never needed to use ARROW_WITH_ZLIB=ON. I have no idea why it became necessary this time.
    I just reproduced the compilation error in a Singularity container. The container was built from this definition file.

    Singularity> cd temp/arrow/cpp/
Singularity> mkdir release
Singularity> cd release/
Singularity> cmake .. -DCMAKE_BUILD_TYPE=Release -DARROW_DEPENDENCY_SOURCE=AUTO -DARROW_FLIGHT=ON -DARROW_BUILD_BENCHMARKS=ON
...
Singularity> make
...
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflateInit2_'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflate'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflateInit2_'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflate'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflateEnd'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflateEnd'
collect2: error: ld returned 1 exit status
src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/build.make:156: recipe for target 'release/arrow-flight-perf-server' failed
make[2]: *** [release/arrow-flight-perf-server] Error 1
CMakeFiles/Makefile2:2648: recipe for target 'src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all' failed
make[1]: *** [src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

    I cloned the latest apache/arrow repo (to temp/arrow) to ensure this problem is not caused by my accidental mess-up to my local arrow repo.
    The latest commit of the temporary clone is:

    Singularity> git log
commit 045fe1382349485e6eb2ce9466757efedf207c94 (HEAD -> master, origin/master, origin/HEAD)

  12. asfimport commented on Jun 11, 2020

    @asfimport
    asfimport
    on Jun 11, 2020
    Author

    Antoine Pitrou / @pitrou:
    @MaChengxin Do you still encounter this issue on git master?

  13. asfimport commented on Jun 14, 2020

    @asfimport
    asfimport
    on Jun 14, 2020
    Author

    Chengxin Ma / @MaChengxin:
    @pitrou The problem is gone.

    I checked with HEAD pointing to 3ccecfb36.
     ```Java

    Singularity> git rev-parse --short HEAD
    3ccecfb

  14. asfimport
    closed this as completedon Jun 14, 2020
  15. asfimport
    assigned
    pitrou
    on Jan 10, 2023
  16. asfimport
    added this to the 1.0.0 milestone on Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Component: BenchmarkingComponent: C++Component: FlightRPCType: bug

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions