Skip to content

lib-dynload installed in wrong location on a 64-bit system when CONFIG_SITE is set #108819

Open
Open
lib-dynload installed in wrong location on a 64-bit system when CONFIG_SITE is set#108819
Labels
OS-linuxtopic-installationtype-bugAn unexpected behavior, bug, or error

Description

@tfpf
tfpf
opened on Sep 2, 2023

Bug report

Checklist

  • I am confident this is a bug in CPython, not a bug in a third-party project
  • I have searched the CPython issue tracker,
    and am confident this bug has not been reported before

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0a0 (heads/main:d4e534cbb3, Sep 2 2023, 21:58:58) [GCC 13.2.1 20230803 [revision cc279d6c64562f05019e1d12d0d825f9391b5553]]

A clear and concise description of the bug:

On openSUSE Tumbleweed 20230823, when CPython is compiled from source,

  • lib-dynload is placed in /usr/local/lib64/python3.13.
  • everything else is placed in /usr/local/lib/python3.13.

As a result, the interpreter is unable to locate some modules like readline (even though readline-devel is installed) and _posixsubprocess.

Here are the exact commands I ran.

$ CFLAGS='-march=native -mtune=native' ./configure --enable-optimizations --with-lto=yes
$ make -j4
$ sudo make -j4 altinstall
$ python3.13
Could not find platform dependent libraries <exec_prefix>
Python 3.13.0a0 (heads/main:d4e534cbb3, Sep  2 2023, 21:58:58) [GCC 13.2.1 20230803 [revision cc279d6c64562f05019e1d12d0d825f9391b5553]] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "/etc/pythonstart", line 7, in <module>
    import readline
ModuleNotFoundError: No module named 'readline'
>>>

Symlinking lib-dynload fixed the problem for me:

$ sudo ln -s /usr/local/lib64/python3.13/lib-dynload /usr/local/lib/python3.13
$ python3.13
Python 3.13.0a0 (heads/main:d4e534cbb3, Sep  2 2023, 21:58:58) [GCC 13.2.1 20230803 [revision cc279d6c64562f05019e1d12d0d825f9391b5553]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

but I believe everything should go in /usr/local/lib, as happens on other Linux distributions I have used.

Linked PRs

Activity

  1. tfpf
    added
    type-bugAn unexpected behavior, bug, or error
    on Sep 2, 2023
  2. sunmy2019
    added
    buildThe build process and cross-build
    on Sep 2, 2023

  3. tfpf commented on Sep 8, 2023

    @tfpf
    tfpf
    on Sep 8, 2023
    Author

    I tried to narrow down the source of the problem. According to this article, /usr/local/share/config.site is sourced if it exists. It does exist on my openSUSE Tumbleweed installation; sourcing it sets libdir to '${exec_prefix}/lib64'. Since CONFIG_SITE is set to /usr/share/site/x86_64-pc-linux-gnu, said file is sourced (this has the effect of setting libdir to '${exec_prefix}/lib64'), and the makefile generated contains, among other things:

    LIBDIR=		${exec_prefix}/lib64
PLATLIBDIR=	lib

    which I think is the reason lib-dynload went in the wrong location:

    BINLIBDEST=	$(LIBDIR)/python$(VERSION)
DESTSHARED=	$(BINLIBDEST)/lib-dynload

    while the remaining files, which depend on PLATLIBDIR (which is hard-coded in configure.ac) went in the expected location.

    The solution is to clear CONFIG_SITE. When I built CPython using:

    CONFIG_SITE="" ./configure
make -j4
sudo make -j4 altinstall

    there were no errors. This mismatch between LIBDIR and PLATLIBDIR when CONFIG_SITE is set looks like a bug in the build process.

  4. tfpf
    changed the title [-]`lib-dynload` installed in wrong location on openSUSE while building CPython[/-] [+]`lib-dynload` installed in wrong location on a 64-bit system when `CONFIG_SITE` is set[/+] on Sep 8, 2023

  5. tfpf commented on Sep 8, 2023

    @tfpf
    tfpf
    on Sep 8, 2023
    Author

    I successfully reproduced the issue using Docker.

    Dockerfile 
    FROM opensuse/tumbleweed
RUN zypper install -y gawk gcc git make zlib-devel
RUN git clone https://github.com/python/cpython.git

RUN mkdir -p /usr/share/site
COPY x86_64-pc-linux-gnu /usr/share/site
ENV CONFIG_SITE=/usr/share/site/x86_64-pc-linux-gnu

WORKDIR cpython
RUN ./configure
RUN make -j8
RUN make -j8 altinstall

CMD ["python3.13", "-c", "print(0)"]
    x86_64-pc-linux-gnu (copied from my system) 
    #!/bin/sh
# Site script for configure. It is resourced via $CONFIG_SITE environment varaible.

# If user did not specify libdir, guess the correct target:
# Use lib64 for 64 bit bi-arch targets, keep the default for the rest.
if test "$libdir" = '${exec_prefix}/lib' ; then

	ac_config_site_64bit_host=NONE

	case "$host" in
	"" )
		# User did not specify host target.
		# The native platform x86_64 is a bi-arch platform.
		# Try to detect cross-compilation to inferior architecture.

		# We are trying to guess 32-bit target compilation. It's not as easy as
		# it sounds, as there is possible several intermediate combinations.
		ac_config_site_cross_to_32bit_host=NONE

		# User defined -m32 in CFLAGS or CXXFLAGS or CC or CXX:
		# (It's sufficient for 32-bit, but alone may cause mis-behavior of some checks.)
		case "$CFLAGS $CXXFLAGS $CC $CXX" in
		*-m32*)
			ac_config_site_cross_to_32bit_host=YES
			;;
		esac

		# Running with linux32:
		# (Changes detected platform, but not the toolchain target.)
		case "`/bin/uname -i`" in
		x86_64 | ppc64 | s390x | aarch64 )
			;;
		* )
			ac_config_site_cross_to_32bit_host=YES
			;;
		esac

		if test "x$ac_config_site_cross_to_32bit_host" = xNONE; then
			ac_config_site_64bit_host=YES
		fi

		;;
	*x86_64* | *ppc64* | *s390x* | *aarch64* )
		ac_config_site_64bit_host=YES
		;;
	esac

	if test "x$ac_config_site_64bit_host" = xYES; then
		libdir='${exec_prefix}/lib64'
	fi
fi

# Continue with the standard behavior of configure defined in AC_SITE_LOAD:
if test "x$prefix" != xNONE; then
	ac_site_files="$prefix/share/config.site $prefix/etc/config.site"
else
	ac_site_files="$ac_default_prefix/share/config.site $ac_default_prefix/etc/config.site"
fi

for ac_site_file in $ac_site_files
do
	case $ac_site_file in #(
	*/*) :
		 ;; #(
	*) :
		ac_site_file=./$ac_site_file ;;
esac
	if test -f "$ac_site_file" && test -r "$ac_site_file"; then
		{ printf "%s\n" "/usr/share/site/x86_64-pc-linux-gnu:${as_lineno-$LINENO}: loading site script $ac_site_file" >&5
printf "%s\n" "/usr/share/site/x86_64-pc-linux-gnu: loading site script $ac_site_file" >&6;}
		sed 's/^/| /' "$ac_site_file" >&5
		. "$ac_site_file" \
			|| { { printf "%s\n" "/usr/share/site/x86_64-pc-linux-gnu:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
printf "%s\n" "/usr/share/site/x86_64-pc-linux-gnu: error: in \`$ac_pwd':" >&2;}
as_fn_error $? "failed to load site script $ac_site_file
See \`config.log' for more details" "$LINENO" 5; }
	fi
done
    docker build . -t gh-108819
docker run -t gh-108819

  6. slburson commented on Jan 9, 2024

    @slburson
    slburson
    on Jan 9, 2024

    Hit the same problem installing 3.11.7 on openSUSE Leap 15.5.

  7. tfpf commented on Jan 12, 2024

    @tfpf
    tfpf
    on Jan 12, 2024
    Author

    Could someone vet this? It looks like the solution would be to place everything in /usr/local/lib, ignoring CONFIG_SITE, but I'm guessing that'll have side-effects.

  8. johnny94 commented on Aug 1, 2024

    @johnny94
    johnny94
    on Aug 1, 2024

    I had the same problem when installing 3.12.4 on Rocky 8 & 9. You can add --with-platlibdir to specify platform-specific library directory. In my case, I use the following script to config building

    ./configure --prefix=/usr --enable-optimizations --with-platlibdir=lib64

    Result

    Python 3.12.4 (main, Aug  1 2024, 12:38:12) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/usr/lib64/python312.zip', '/usr/lib64/python3.12', '/usr/lib64/python3.12/lib-dynload', '/usr/lib64/python3.12/site-packages', '/usr/lib/python3.12/site-packages']
>>> print(sys.platlibdir)
lib64

    Not necessary to create symlink for lib-dynload anymore.

  9. picnixz
    added
    topic-installation
    OS-linux
    and removed
    buildThe build process and cross-build
    on Apr 28, 2025
  10. FFY00
    added a commit that references this issue on Apr 29, 2025

    pythonGH-108819: fix LIBDEST not honoring --with-platlibdir

    6452907

  11. FFY00 commented on Apr 29, 2025

    @FFY00
    FFY00
    on Apr 29, 2025
    Member

    @tfpf, thank you for looking into this issue, I agree that your issue is likely due to a mismatch between the values used for libdir and platlibdir, which may be introduced by setting libdir to something other than ${exec_prefix}/lib in config.site.

    If it isn't, that, at least, is an issue, given the current getpath.py code. I have opened GH-133163 with a fix for that, which I think should solve it on your side too.

  12. FFY00
    added a commit that references this issue on Dec 9, 2025

    GH-108819: fix LIBDEST not honoring --with-platlibdir (#133163)

    7f497c3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-linuxtopic-installationtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions