Skip to content

Incorrect installation of lib-dynload for custom builds on openSUSE #131425

Description

@intel-hm

Bug report

Bug description:

from Python-3.13.2.tar.xz

./configure --prefix=$RUN/python/3.13.2
make -j 16
make -j 4 install

dynamic libraries are installed to $RUN/3.13.2/lib64/python3.13/lib-dynload
but internally sys.path uses $RUN/3.13.2/lib/python3.13/lib-dynload

setting a link resolves the

errors that show up:

> python3
**Could not find platform dependent libraries <exec_prefix>**
Python 3.13.2 (main, Mar 18 2025, 10:25:05) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
**warning: can't use pyrepl: No module named '_opcode'**

pip3 download pip wheel setuptools 
...
    from _posixsubprocess import fork_exec as _fork_exec
**ModuleNotFoundError: No module named '_posixsubprocess'**

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Activity

  1. picnixz commented on Mar 18, 2025

    @picnixz
    Member

    As an openSUSE user, I had this problem as well. I think I did a combination of --exec-prefix=... and altinstall. I can't remember exactly the command and I won't be back until Sunday. I can check at that time.

  2. changed the title [-]installation form source on "openSUSE Leap 15.5" and "SUSE Linux Enterprise Server 15 SP6" - incorrect location of python3.13/lib-dynload[/-] [+]Incorrect installation of `lib-dynload` for custom builds on openSUSE[/+] on Mar 18, 2025
  3. tfpf commented on Mar 22, 2025

    @tfpf

    This was previously reported and discussed in #108819. There are workarounds suggested in the comments on that issue, but no comments from any maintainers.

  4. picnixz commented on Mar 22, 2025

    @picnixz
    Member

    I think I actually also specified the libdir explicitly so that it doesn't use lib64 but lib instead. I'll see if I can write better installation instructions for openSUSE

  5. intel-hm commented on Mar 22, 2025

    @intel-hm
    Author

    I think that's an error in the configure script. And it's not only OPENSUSE - it's also SLES

  6. picnixz commented on Mar 22, 2025

    @picnixz
    Member

    Yes I think so as well but I'll need to confirm it properly. I'll also tag those responsible for openSUSE contact

  7. picnixz commented on Apr 28, 2025

    @picnixz
    Member

    @mcepl From a configure PoV, what's the best? detect that we're on openSUSE or SLES or do something else?

  8. intel-hm commented on Apr 28, 2025

    @intel-hm
    Author

    could someone figure out what causes this difference?
    dynamic libraries are installed to $RUN/3.13.2/lib64/python3.13/lib-dynload
    but internally sys.path uses $RUN/3.13.2/lib/python3.13/lib-dynload

    how are those 2 paths defined in configure? what is different on SUSE that causes the difference?

  9. tfpf commented on Apr 28, 2025

    @tfpf

    could someone figure out what causes this difference? dynamic libraries are installed to $RUN/3.13.2/lib64/python3.13/lib-dynload but internally sys.path uses $RUN/3.13.2/lib/python3.13/lib-dynload

    how are those 2 paths defined in configure? what is different on SUSE that causes the difference?

    I believe it’s because the configure script doesn’t handle the possibility that CONFIG_SITE may be set. I’m pasting a part of my analysis from #108819.

    Since CONFIG_SITE is set to /usr/share/site/x86_64-pc-linux-gnu, said file is sourced (this has the effect of setting libdir to '${exec_prefix}/lib64'), and the makefile generated contains, among other things:

    LIBDIR=		${exec_prefix}/lib64
    PLATLIBDIR=	lib
    

    which I think is the reason lib-dynload went in the wrong location:

    BINLIBDEST=	$(LIBDIR)/python$(VERSION)
    DESTSHARED=	$(BINLIBDEST)/lib-dynload
    

    while the remaining files, which depend on PLATLIBDIR (which is hard-coded in configure.ac) went in the expected location.

  10. FFY00 commented on Apr 29, 2025

    @FFY00
    Member

    I opened GH-133163, which might fix this issue. @intel-hm, if you could test it, it'd be appreciated.

  11. intel-hm commented on Apr 29, 2025

    @intel-hm
    Author

    if you give me the right git commands to download the correct files (my git knowledge is barely existant....)

  12. FFY00 commented on Apr 29, 2025

    @FFY00
    Member

    if you give me the right git commands to download the correct files (my git knowledge is barely existant....)

    In that case, I'd recommend you use the Github CLI, which simplifies this quite significantly 😊

    Using it, it should be as straightforward as:

    $ gh pr checkout 133163

    Otherwise, you'll have to manually add a remote for my fork, and then checkout the correct branch.

    $ git remote add ffy00 https://github.com/FFY00/cpython.git
    $ git fetch ffy00
    $ git checkout gh-108819
  13. intel-hm commented on Apr 29, 2025

    @intel-hm
    Author

    wouldn't I also need a git clone? Github CLI is not really an option

    $ git clone https://github.com/FFY00/cpython.git
    $ git remote add ffy00 https://github.com/FFY00/cpython.git
    $ git fetch ffy00
    $ git checkout gh-108819

  14. FFY00 commented on Apr 29, 2025

    @FFY00
    Member

    You can either clone the main repo and add my fork as a remote, or clone my fork directly, then you can checkout my gh-108819 branch.

  15. mcepl commented on Apr 29, 2025

    @mcepl
    Contributor

    I obviously don’t see anything like this with our system packages:

    Python 3.13.3 (main, Apr 11 2025, 19:56:42) [GCC]
    Type 'copyright', 'credits' or 'license' for more information
    IPython 8.31.0 -- An enhanced Interactive Python. Type '?' for help.
    In [1]: import sys
    
    In [2]: sys.path
    Out[2]: 
    ['/usr/bin',
     '/usr/lib64/python313.zip',
     '/usr/lib64/python3.13',
     '/usr/lib64/python3.13/lib-dynload',
     '',
     '/home/matej/.local/lib/python3.13/site-packages',
     '/usr/lib/python3.13/site-packages/setuptools/_vendor',
     '/home/matej/archiv/knihovna/repos/osc',
     '/home/matej/archiv/2024/SUSE/projekty/maintenance-toolkit',
     '/home/matej/archiv/knihovna/repos/epy/src',
     '/usr/lib64/python3.13/site-packages',
     '/usr/lib64/python3.13/site-packages/PIL',
     '/usr/lib64/python3.13/_import_failed',
     '/usr/lib/python3.13/site-packages']
    
    In [3]: 
    

    However, we have plenty of patches and configuration. Perhaps looking at https://build.opensuse.org/package/show/devel:languages:python:Factory/python313 would show something interesting. However, to be honest, I don’t care that much about user-build builds; I think we are doing pretty good job in packaging Python ourselves and we support those.

  16. FFY00 commented on Apr 29, 2025

    @FFY00
    Member

    That'd be because you set --with-platlibdir to match the system libdir, so no mismatch would happen there.

  17. intel-hm commented on Apr 29, 2025

    @intel-hm
    Author

    looks good:

    git clone https://github.com/python/cpython.git
    cd cpython/
    git remote add ffy00 https://github.com/FFY00/cpython.git
    git fetch ffy00
    git checkout gh-108819
    
    ./configure --prefix=/lfs/lfs19/python/3.12.7
    make
    make install
    
    ~/cpython> find /lfs/lfs19/python/3.12.7/ -name lib-dynload
    /lfs/lfs19/python/3.12.7/lib/python3.14/lib-dynload
    
    ~/cpython> /lfs/lfs19/python/3.12.7/bin/python3
    Python 3.14.0a7+ (heads/gh-108819:a2b43c9d137, Apr 29 2025, 12:15:56) [GCC 7.5.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from _posixsubprocess import fork_exec as _fork_exec
    >>>
    
    
  18. picnixz commented on Jun 14, 2025

    @picnixz
    Member

    FTR, I usually solve the installation issue by supplying --with-platlibdir=lib64 since the default one falls back to lib instead of lib64.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions