Skip to content

[R] R package is linking to the wrong OpenSSL on MacOS BigSur? #36456

New issue
New issue
Closed
#36551
Closed
[R] R package is linking to the wrong OpenSSL on MacOS BigSur?#36456
#36551
Assignees
kou
Labels
Component: RPriority: BlockerMarks a blocker for the releaseType: bug
Milestone
13.0.0

Description

@paleolimbot
paleolimbot
opened on Jul 4, 2023

Describe the bug, including details regarding any error messages, version, and platform.

From the r-binary-packages job failures: https://github.com/ursacomputing/crossbow/actions/runs/5442353558/jobs/9898961393

All of them look like this:

** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
ERROR: loading failed
 unable to load shared object '/Users/voltrondata/Library/R/arm64/4.1/library/00LOCK-arrow/00new/arrow/libs/arrow.so':
  dlopen(/Users/voltrondata/Library/R/arm64/4.1/library/00LOCK-arrow/00new/arrow/libs/arrow.so, 6): Symbol not found: _EVP_DigestSignUpdate
* removing ‘/Users/voltrondata/Library/R/arm64/4.1/library/arrow’
  Referenced from: /Users/voltrondata/Library/R/arm64/4.1/library/00LOCK-arrow/00new/arrow/libs/arrow.so
  Expected in: flat namespace
 in /Users/voltrondata/Library/R/arm64/4.1/library/00LOCK-arrow/00new/arrow/libs/arrow.so
Error: loading failed
Execution halted
Warning message:
In install.packages("arrow", type = "source", repos = sub("file://",  :
  installation of package ‘arrow’ had non-zero exit status
Error in library(arrow) : there is no package called ‘arrow’
Execution halted

Component(s)

R

Activity

  1. paleolimbot
    added
    Type: bug
    on Jul 4, 2023
  2. github-actions
    added
    Component: R
    on Jul 4, 2023

  3. paleolimbot commented on Jul 4, 2023

    @paleolimbot
    paleolimbot
    on Jul 4, 2023
    MemberAuthor

    I mention OpenSSL here because the symbol that is not found is EVP_DigestSignUpdate...I suppose it could also be failing to link to any OpenSSL.

  4. paleolimbot commented on Jul 4, 2023

    @paleolimbot
    paleolimbot
    on Jul 4, 2023
    MemberAuthor

    I wonder if:

    arrow/r/configure

    Lines 111 to 120 in 7ecec6c

    # find openssl on macos. macOS ships with libressl. openssl is installable
    # with brew, but it is generally not linked. We can over-ride this and find
    # openssl but setting OPENSSL_ROOT_DIR (which cmake will pick up later in
    # the installation process). FWIW, arrow's cmake process uses this
    # same process to find openssl, but doing it now allows us to catch it in
    # nixlibs.R and throw a nicer error.
    if [ "${OPENSSL_ROOT_DIR}" = "" ] && brew --prefix openssl >/dev/null 2>&1; then
    export OPENSSL_ROOT_DIR="`brew --prefix openssl`"
    export PKG_CONFIG_PATH="${OPENSSL_ROOT_DIR}/lib/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}"
    fi

    is at odds with:

    arrow/cpp/cmake_modules/FindOpenSSLAlt.cmake

    Lines 22 to 39 in 7ecec6c

    if(APPLE AND NOT OPENSSL_ROOT_DIR)
    find_program(BREW brew)
    if(BREW)
    foreach(BREW_OPENSSL_VERSION "" "3" "3.0" "1.1")
    set(BREW_OPENSSL_PACKAGE "openssl")
    if(BREW_OPENSSL_VERSION)
    string(APPEND BREW_OPENSSL_PACKAGE "@${BREW_OPENSSL_VERSION}")
    endif()
    execute_process(COMMAND ${BREW} --prefix --installed ${BREW_OPENSSL_PACKAGE}
    OUTPUT_VARIABLE BREW_OPENSSL_PREFIX
    OUTPUT_STRIP_TRAILING_WHITESPACE)
    if(BREW_OPENSSL_PREFIX)
    set(OPENSSL_ROOT_DIR ${BREW_OPENSSL_PREFIX})
    break()
    endif()
    endforeach()
    endif()
    endif()

    (that was added in #36336 )

  5. kou commented on Jul 5, 2023

    @kou
    kou
    on Jul 5, 2023
    Member

    Ah, this is caused by mixing multiple OpenSSL versions.
    It seems that openssl/*.h of OpenSSL 1.1 is used but libcrypto.dylib of OpenSSL 3 is used when building Apache Arrow C++. (Is autobrew used to build Apache Arrow C++?)
    Could you check it?

  6. paleolimbot commented on Jul 5, 2023

    @paleolimbot
    paleolimbot
    on Jul 5, 2023
    MemberAuthor

    Yes, it's autobrew in that job ( https://github.com/ursacomputing/crossbow/actions/runs/5442353558/jobs/9897977884#step:13:60 ). I think that script is here: https://github.com/apache/arrow/blob/main/r/tools/autobrew and the formula is here: https://github.com/apache/arrow/blob/main/dev/tasks/homebrew-formulae/autobrew/apache-arrow-static.rb .

    I think I'm remembering correctly that we always try to link to system openssl on MacOS and not the one provided by homebrew/autobrew... @assignUser may be more familiar with the details!

    Also worth nothing that we're considering dropping autobrew to reduce the number of steps needed for the R package release; however, I don't know if doing a bundled build (i.e., running cmake on the bundled cpp source instead of autobrew) is more or less complicated with respect to openssl.

  7. assignUser commented on Jul 6, 2023

    @assignUser
    assignUser
    on Jul 6, 2023
    Member

    may be more familiar with the details

    eh, not really due to lack of macos locally ^^ but I can also have a look at this!

  8. raulcd commented on Jul 6, 2023

    @raulcd
    raulcd
    on Jul 6, 2023
    Member

    homebrew-r-autobrew nightly job seems to fail with the same issue. Should this be considered a blocker for the release? I am marking it as a blocker at the moment.

  9. raulcd
    added this to the 13.0.0 milestone on Jul 6, 2023
  10. raulcd
    added
    Priority: BlockerMarks a blocker for the release
    on Jul 6, 2023

  11. paleolimbot commented on Jul 6, 2023

    @paleolimbot
    paleolimbot
    on Jul 6, 2023
    MemberAuthor

    Thanks...yes, it's a blocker for release! CRAN does some builds on MacOS BigSur which is how most people will install the R package.

  12. github-actions
    assigned
    kou
    on Jul 7, 2023

  13. assignUser commented on Jul 7, 2023

    @assignUser
    assignUser
    on Jul 7, 2023
    Member

    As discussed in the PR this seems to be an issue with the gh runners explicitly running brew link openssl which causes the isssue. It would be nice if someone could try to repo locally with a default brew openssl install to make sure. I would assume that cran doesn't do this...

    edit: or not as that solution does not work afterall

  14. kou
    added a commit that references this issue on Jul 7, 2023

    apacheGH-36456: [R] Remove system OpenSSL to avoid mixing OpenSSL ver…

    f28684d

  15. 2 remaining items

  16. assignUser commented on Jul 7, 2023

    @assignUser
    assignUser
    on Jul 7, 2023
    Member

    In the r-binary-packages job we modify the formula with the current hash and repo, maybe that is what's missing? There is some jinja in there but this is what we do, cwd is arrow/r:

     cp ../dev/tasks/homebrew-formulae/autobrew/apache-arrow*.rb tools/

      # Pin the git commit in the formula to match
      cd tools
      if [ "{{ is_fork }}" == "true" ]; then
        sed -i.bak -E -e 's/apache\/arrow.git"$/{{ arrow.github_repo.split("/") | join("\/") }}.git", :revision => "'"{{ arrow.head }}"'"/' apache-arrow*.rb
      else
        sed -i.bak -E -e 's/arrow.git"$/arrow.git", :revision => "'"{{ arrow.head }}"'"/' apache-arrow*.rb
      fi
      rm -f apache-arrow*.rb.bak

  17. paleolimbot commented on Jul 10, 2023

    @paleolimbot
    paleolimbot
    on Jul 10, 2023
    MemberAuthor

    Hmm...from the logs we also get ( https://github.com/ursacomputing/crossbow/actions/runs/5442353558/jobs/9897978025#step:13:5350 )

    -- Check for working CXX compiler: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow-static/Library/Homebrew/shims/mac/super/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Note at CMakeLists.txt:66 (message):
  The Google Cloud C++ client libraries use the native OpenSSL library.  In
  most macOS systems, you need to set the OPENSSL_ROOT_DIR environment
  variable to find this dependency, for example:

      cmake -DOPENSSL_ROOT_DIR=/usr/local/opt/openssl ...

  You have not set this variable.  Most likely, this will result in a broken
  build with fairly obscure error messages.  If your environment does not
  require setting OPENSSL_ROOT_DIR, you can disable this check using:

      cmake -DGOOGLE_CLOUD_CPP_ENABLE_MACOS_OPENSSL_CHECK=OFF ...

    I see set(OPENSSL_ROOT_DIR ${BREW_OPENSSL_PREFIX}) in the current version of "find openssl"...somehow that isn't propagating to the Google Cloud Platform sdk?

  18. nealrichardson commented on Jul 12, 2023

    @nealrichardson
    nealrichardson
    on Jul 12, 2023
    Member
    • Recreating a local autobrew build is hard. The comment at https://github.com/apache/arrow/blob/main/r/configure#L241 suggests that you can do cp ../dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb tools/apache-arrow.rb. I've done this before successfully but currently it gives me an error (see details).

    Now that there are two autobrew formulae, I think you have to copy them both, i.e. also apache-arrow-static.rb. That's the one that will get used on a contemporary version of macOS. See r/tools/autobrew

  19. paleolimbot
    closed this as completedin #36551on Jul 16, 2023
  20. paleolimbot
    added a commit that references this issue on Jul 16, 2023

    GH-36456: [R] Link to correct version of OpenSSL when using autobrew (#…

    ed87a5b
  21. paleolimbot
    modified the milestones: 13.0.0, 14.0.0 on Jul 16, 2023

  22. raulcd commented on Jul 17, 2023

    @raulcd
    raulcd
    on Jul 17, 2023
    Member

    This is expected to be on 13.0.0, right?

  23. paleolimbot commented on Jul 17, 2023

    @paleolimbot
    paleolimbot
    on Jul 17, 2023
    MemberAuthor

    We don't technically need this in 13.0.0 to release the R package but it will probably make your life easier because it fixes the binary packages and autobrew jobs.

  24. raulcd
    modified the milestones: 14.0.0, 13.0.0 on Jul 17, 2023

  25. raulcd commented on Jul 17, 2023

    @raulcd
    raulcd
    on Jul 17, 2023
    Member

    Thanks @paleolimbot , I'll cherry-pick it

  26. raulcd
    added a commit that references this issue on Jul 17, 2023

    GH-36456: [R] Link to correct version of OpenSSL when using autobrew (#…

    1d5f4cd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Component: RPriority: BlockerMarks a blocker for the releaseType: bug

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions