Skip to content

spin build should fix the rpath for shared libaries #176

Closed
Closed
spin build should fix the rpath for shared libaries#176

Description

@oscarbenjamin
oscarbenjamin
opened on Apr 4, 2024

I'm migrating python-flint to meson-python and testing out spin as a frontend (flintlib/python-flint#129).

The setup that I use with python-flint is to build libraries with --prefix=$(pwd)/.local so that I have isolated builds of the C dependencies inside the python-flint development checkout.

When building wheels with meson-python I can set PKG_CONFIG_PATH to find the libraries and then meson-python fixes the rpath:
https://github.com/mesonbuild/meson-python/blob/c9d1c2dafb54692ad56dcda1bc67fd52b83b29c0/mesonpy/__init__.py#L431-L440

This means that I can do e.g.

$ PKG_CONFIG_PATH=$(pwd)/.local/lib/pkgconfig spin install
 $ pip install --no-build-isolation --editable .
 ...
Successfully installed python-flint-0.6.0
$ python -m flint.test -qt
Running tests...
flint.test: all 49 tests passed!
----------------------------------------
OK: Your installation of python-flint seems to be working just fine!
----------------------------------------

However spin build does not fix the rpath and so spin test does not work unless I set LD_LIBRARY_PATH:

$ PKG_CONFIG_PATH=$(pwd)/.local/lib/pkgconfig spin build --clean
...
[90/90] Linking target src/flint/functions/showgood.cpython-312-x86_64-linux-gnu.so
 $ meson install --only-changed -C build --destdir ../build-install
$ spin test
Invoking `build` prior to running tests:
 $ meson compile -C build
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /home/oscar/current/active/python-flint/build
ninja: Entering directory `/home/oscar/current/active/python-flint/build'
ninja: no work to do.
 $ meson install --only-changed -C build --destdir ../build-install
 $ export PYTHONPATH="/home/oscar/current/active/python-flint/build-install/usr/lib/python3.12/site-packages"
 $ /home/oscar/.pyenv/versions/3.12.0/envs/python-flint-3.12/bin/python3.12 -P -c 'import flint'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/oscar/current/active/python-flint/build-install/usr/lib/python3.12/site-packages/flint/__init__.py", line 1, in <module>
    from .pyflint import *
  File "pyflint.pyx", line 1, in init flint.pyflint
  File "flint_base.pyx", line 1, in init flint.flint_base.flint_base
ImportError: libflint.so.19: cannot open shared object file: No such file or directory
As a sanity check, we tried to import flint.
Stopping. Please investigate the build error.
$ LD_LIBRARY_PATH=$(pwd)/.local/lib spin test
Invoking `build` prior to running tests:
 $ meson compile -C build
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /home/oscar/current/active/python-flint/build
ninja: Entering directory `/home/oscar/current/active/python-flint/build'
ninja: no work to do.
 $ meson install --only-changed -C build --destdir ../build-install
 $ export PYTHONPATH="/home/oscar/current/active/python-flint/build-install/usr/lib/python3.12/site-packages"
 $ /home/oscar/.pyenv/versions/3.12.0/envs/python-flint-3.12/bin/python3.12 -P -c 'import flint'
 $ export PYTHONPATH="/home/oscar/current/active/python-flint/build-install/usr/lib/python3.12/site-packages"
 $ cd /home/oscar/current/active/python-flint/build-install/usr/lib/python3.12/site-packages
 $ /home/oscar/.pyenv/versions/3.12.0/envs/python-flint-3.12/bin/python3.12 -m pytest --rootdir=/home/oscar/current/active/python-flint/build-install/usr/lib/python3.12/site-packages flint
======================================== test session starts ========================================
platform linux -- Python 3.12.0, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/oscar/current/active/python-flint/build-install/usr/lib/python3.12/site-packages
plugins: doctestplus-1.2.1, cov-4.1.0, hypothesis-6.88.4, xdist-3.4.0
collected 49 items                                                                                  

flint/test/test_stuff.py .................................................                    [100%]

======================================== 49 passed in 3.86s =========================================

I think that spin build should set the rpath somehow but I'm not sure where is the place that this should be fixed. I assume that it would be out of place for spin to manipulate the rpath entries directly like meson-python does.

Should meson set the rpath?

Should spin ask meson or meson-python to do it somehow?

Activity

  1. rgommers commented on Apr 4, 2024

    @rgommers
    rgommers
    on Apr 4, 2024
    Contributor

    Since editable installs already work for you, gh-155 to make editable installs work with spin should be helpful here.

    For the default spin build behavior, I'm actually not 100% sure what is happening.

    When building wheels with meson-python I can set PKG_CONFIG_PATH to find the libraries and then meson-python fixes the rpath:

    This is probably not the relevant code, because it's about internal shared libraries, while the ones you have are external (not built as part of the meson setup / meson compile invocation as a subproject) ones, right?

    Normally Meson will take care of rpath rewriting during install. It's possible that that doesn't work for in-tree shared libraries because they're not considered external. Does it work without LD_LIBRARY_PATH if you place them elsewhere, like $(pwd)/../.local/lib?

    The entries should have absolute paths for shared library dependencies that are not on the default loader search path. For example:

    # spin build folder
$ ldd build/scipy/linalg/_fblas.cpython-311-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007ffeee2fd000)
        libopenblas.so.0 => /home/rgommers/mambaforge/envs/scipy-dev/lib/libopenblas.so.0 (0x000078fd15e00000)
        libc.so.6 => /usr/lib/libc.so.6 (0x000078fd15bfb000)
        libm.so.6 => /usr/lib/libm.so.6 (0x000078fd15b0f000)

# spin install folder
$ ldd build-install/lib/python3.11/site-packages/scipy/linalg/_fblas.cpython-311-x86_64-linux-gnu.so 
        linux-vdso.so.1 (0x00007ffcb6baa000)
        libopenblas.so.0 => /home/rgommers/mambaforge/envs/scipy-dev/lib/libopenblas.so.0 (0x00007311fc600000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007311fc41e000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007311fc332000)

    RPATH entry:

    $ readelf -d build/scipy/linalg/_fblas.cpython-311-x86_64-linux-gnu.so | rg RPATH
 0x000000000000000f (RPATH)              Library rpath: [/home/rgommers/mambaforge/envs/scipy-dev/lib]

  2. oscarbenjamin commented on Apr 4, 2024

    @oscarbenjamin
    oscarbenjamin
    on Apr 4, 2024
    Author

    the ones you have are external (not built as part of the meson setup / meson compile invocation as a subproject) ones, right?

    Yes, that's correct.

    This is what I get from ldd:

    # build dir
$ ldd build/src/flint/types/fmpz.cpython-312-x86_64-linux-gnu.so
	linux-vdso.so.1 (0x00007ffd673d8000)
	libflint.so.19 => /home/oscar/current/active/python-flint/build/src/flint/types/../../../../.local/lib/libflint.so.19 (0x0000737a77000000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x0000737a76c00000)
	libmpfr.so.6 => /home/oscar/current/active/python-flint/.local/lib/libmpfr.so.6 (0x0000737a76800000)
	libgmp.so.10 => /home/oscar/current/active/python-flint/.local/lib/libgmp.so.10 (0x0000737a76f5b000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x0000737a76e74000)
	/lib64/ld-linux-x86-64.so.2 (0x0000737a77c8f000)

# install dir
$ ldd build-install/usr/lib/python3.12/site-packages/flint/types/fmpz.cpython-312-x86_64-linux-gnu.so 
	linux-vdso.so.1 (0x00007ffef73fd000)
	libflint.so.19 => not found
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x0000709664000000)
	/lib64/ld-linux-x86-64.so.2 (0x000070966440b000)

    This is what readelf shows:

    $ readelf -d build/src/flint/types/fmpz.cpython-312-x86_64-linux-gnu.so

Dynamic section at offset 0x41cc8 contains 26 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libflint.so.19]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../../../../.local/lib]
 0x000000000000000c (INIT)               0x6000
 0x000000000000000d (FINI)               0x37de4
 0x0000000000000019 (INIT_ARRAY)         0x42cb8
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x42cc0
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x2f0
 0x0000000000000005 (STRTAB)             0x1878
 0x0000000000000006 (SYMTAB)             0x318
 0x000000000000000a (STRSZ)              4012 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x43000
 0x0000000000000002 (PLTRELSZ)           4392 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x4ad8
 0x0000000000000007 (RELA)               0x2a20
 0x0000000000000008 (RELASZ)             8376 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x29f0
 0x000000006fffffff (VERNEEDNUM)         1
 0x000000006ffffff0 (VERSYM)             0x2824
 0x000000006ffffff9 (RELACOUNT)          306
 0x0000000000000000 (NULL)               0x0
$ readelf -d build-install/usr/lib/python3.12/site-packages/flint/types/fmpz.cpython-312-x86_64-linux-gnu.so 

Dynamic section at offset 0x41cc8 contains 25 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libflint.so.19]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000c (INIT)               0x6000
 0x000000000000000d (FINI)               0x37de4
 0x0000000000000019 (INIT_ARRAY)         0x42cb8
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x42cc0
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x2f0
 0x0000000000000005 (STRTAB)             0x1878
 0x0000000000000006 (SYMTAB)             0x318
 0x000000000000000a (STRSZ)              4012 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x43000
 0x0000000000000002 (PLTRELSZ)           4392 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x4ad8
 0x0000000000000007 (RELA)               0x2a20
 0x0000000000000008 (RELASZ)             8376 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x29f0
 0x000000006fffffff (VERNEEDNUM)         1
 0x000000006ffffff0 (VERSYM)             0x2824
 0x000000006ffffff9 (RELACOUNT)          306
 0x0000000000000000 (NULL)               0x0

    I'm not sure why I have an extra usr like build-install/usr/.... When building with spin build I get:

      User defined options
    prefix: /usr

    I don't think I've set that anywhere though...

  3. rgommers commented on Apr 4, 2024

    @rgommers
    rgommers
    on Apr 4, 2024
    Contributor

    Library runpath: [$ORIGIN/../../../../.local/lib]

    Yep, this being a relative rather than an absolute path should be the problem.

    I don't think I've set that anywhere though...

    That's the output of meson, so "user defined" here is something set by spin.

  4. oscarbenjamin commented on Apr 4, 2024

    @oscarbenjamin
    oscarbenjamin
    on Apr 4, 2024
    Author

    Does it work without LD_LIBRARY_PATH if you place them elsewhere, like $(pwd)/../.local/lib?

    No, it still doesn't work in that case. I just get an extra ..:

     0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../../../../../.local/lib]

  5. oscarbenjamin commented on Apr 4, 2024

    @oscarbenjamin
    oscarbenjamin
    on Apr 4, 2024
    Author

    No, it still doesn't work in that case. I just get an extra ..:

    To be clear I used an absolute path to a directory outside of the working directory

    $ PKG_CONFIG_PATH=/home/oscar/current/active/.local/lib/pkgconfig/ spin build --clean

    but it somehow got turned into a relative path.

  6. oscarbenjamin commented on Apr 4, 2024

    @oscarbenjamin
    oscarbenjamin
    on Apr 4, 2024
    Author

    So cutting out spin then the problem is just:

    $ PKG_CONFIG_PATH=$(pwd)/.local/lib/pkgconfig meson setup build
$ meson compile -C build
$ readelf -d build/src/flint/types/fmpz.cpython-312-x86_64-linux-gnu.so | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../../../../.local/lib]

    We wanted an absolute path to be in the binary but we've been left with a relative one.

    Should that be considered a bug in meson?

  7. oscarbenjamin commented on Apr 4, 2024

    @oscarbenjamin
    oscarbenjamin
    on Apr 4, 2024
    Author

    It all works fine on macos so I'm only seeing this problem on Linux.

  8. stefanv commented on May 1, 2024

    @stefanv
    stefanv
    on May 1, 2024
    Member

    @oscarbenjamin If I understand this issue correctly, this is a Meson problem. If you agree, we can close this, otherwise I can investigate further.

  9. oscarbenjamin commented on May 1, 2024

    @oscarbenjamin
    oscarbenjamin
    on May 1, 2024
    Author

    Yeah, this is just a meson issue I think.

  10. oscarbenjamin
    closed this as completedon May 1, 2024
  11. oscarbenjamin
    reopened this on May 8, 2024
  12. oscarbenjamin
    closed this as completedon May 8, 2024

  13. oscarbenjamin commented on May 8, 2024

    @oscarbenjamin
    oscarbenjamin
    on May 8, 2024
    Author

    The solution to this (mesonbuild/meson#13046 (comment)) is to set LDFLAGS when calling spin build i.e.:

    PKG_CONFIG_PATH=$(pwd)/.local/lib/pkgconfig LDFLAGS=-Wl,-rpath=$(pwd)/.local/lib spin build

    Possibly it would be useful for spin to add a more discoverable CLI option for that like

    spin build --local-lib-dir=.local/lib

    In flintlib/python-flint#135 I added a custom option spin build --local-flint=.local/lib/pkgconfig which translates to

    spin build -- --pkg-config-path=$(pwd)/.local/lib/pkgconfig -Dlocal_flint=true

    The local_flint project option sets the linker flags to install the rpaths.

  14. oscarbenjamin commented on May 8, 2024

    @oscarbenjamin
    oscarbenjamin
    on May 8, 2024
    Author

    From the discussion in mesonbuild/meson#13046 I think that this is sort of a spin-specific situation because of the fact that we are "installing" things into a local directory.

  15. rgommers commented on May 8, 2024

    @rgommers
    rgommers
    on May 8, 2024
    Contributor

    I think that this is sort of a spin-specific situation because of the fact that we are "installing" things into a local directory.

    I think it's specific to "build against a shared library that's not on the default loader search path", and is unrelated to the install location. I.e., whether you use spin or use for example pip install . should have the same failure mode (editable installs do not though, but regular installs will).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions