--with-system-libmpdec / --with-system-expat still uses the vendored headers

Bug report

Python 3.12.0a1 or the main branch. Run:

$ sudo dnf install mpdecimal-devel expat-devel  # or similar for your OS
...
$ git clean -fdx
$ rm -r Modules/_decimal/libmpdec
$ rm -r Modules/expat
$ ./configure --config-cache --enable-shared --with-system-libmpdec --with-system-expat
$ make
...
make: *** No rule to make target 'Modules/_decimal/libmpdec/basearith.h', needed by 'Modules/_decimal/_decimal.o'.  Stop.

$ git restore Modules/_decimal/libmpdec
$ make
...
make: *** No rule to make target 'Modules/expat/ascii.h', needed by 'Modules/pyexpat.o'.  Stop.

This works fine with Python 3.11.0. I suspect somewhere along #94474 or a similar change, this has changed behavior, but I have not tried bisecting yet.

Your environment

CPython versions tested on: 3.12.0a1 or the main branch
Operating system and architecture: Fedora Linux x86_64

PR: gh-98707: Don't let --with-system-libmpdec / --with-system-expat use the vendored headers #98711

PR: [3.11] gh-98707: configure --with-system-libmpdec and --with-system-expat no longer include vendored headers (GH-98711) #99391

Bisecting with only Modules/_decimal/libmpdec removed, because I got some other failures with removed Modules/expat on the way.

EDIT: Apparently, that's what you get when you spell expat as expact.

@tiran

Indeed, my gut feeling was correct:

81dca70 is the first new commit
commit 81dca70
Author: Christian Heimes @tiran
Date: Thu Jul 14 09:51:49 2022 +0200

gh-93939: Build C extensions without setup.py (GH-94474)

I am really really bad at autotools, but my best guess is that this is caused by:

cpython/Makefile.pre.in

Line 2584 in de69816

MODULE_PYEXPAT_DEPS=$(LIBEXPAT_HEADERS) @LIBEXPAT_INTERNAL@

As LIBEXPAT_HEADERS contains a list that starts with Modules/expat/ascii.h.

cpython/Makefile.pre.in

Line 2589 in de69816

    
           MODULE__DECIMAL_DEPS=$(srcdir)/Modules/_decimal/docstrings.h $(LIBMPDEC_HEADERS) @LIBMPDEC_INTERNAL@

As LIBMPDEC_HEADERS contains a list that starts with Modules/_decimal/libmpdec/basearith.h.

In both cases, the dependency needs to be made conditional on the relevant --with-system-xxx option.

Testing a possible fix.

Don't let --with-system-libmpdec / --with-system-expat use the vendor…

pythongh-98707: Don't let --with-system-libmpdec / --with-system-expa…

FTR this was a problem on older Pythons as well, but it was not exposed. #94474 did not introduce it, but it exposed it. Another problem this will most likely fix is:

--- STDERR ---
/tmp/tmpt5xmo3n_/cpython/Objects/obmalloc.c:1449:1: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
 1449 | arena_map_get(block *p, int create)
      | ^~~~~~~~~~~~~
make: *** No rule to make target '/tmp/tmpt5xmo3n_/cpython/Modules/_decimal/libmpdec/basearith.c', needed by 'Modules/_decimal/libmpdec/basearith.o'.  Stop.
make: *** Waiting for unfinished jobs....
---- END ----
ERROR
======================================================================
ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.11.0/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0/Tools/freeze/test/freeze.py", line 159, in prepare
    _run_quiet([MAKE, '-C', builddir, '-j8'])
  File "/builddir/build/BUILD/Python-3.11.0/Tools/freeze/test/freeze.py", line 25, in _run_quiet
    return subprocess.run(
           ^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0/Lib/subprocess.py", line 569, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpt5xmo3n_/python-build', '-j8']' returned non-zero exit status 2.
----------------------------------------------------------------------
Ran 1 test in 70.677s
FAILED (errors=1)

EDIT: It does not fix it.

pythongh-98707: Don't let --with-system-libmpdec / --with-system-expa…

00389: Don't let --with-system-libmpdec / --with-system-expat use the…

gh-98707: configure --with-system-libmpdec and --with-system-expat no…

[3.11] pythongh-98707: configure --with-system-libmpdec and --with-sy…

pythongh-98707: configure --with-system-libmpdec and --with-system-ex…

[3.11] gh-98707: configure --with-system-libmpdec and --with-system-e…

Navigation Menu

Repository navigation

--with-system-libmpdec / --with-system-expat still uses the vendored headers #98707

Bug report

Your environment

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

--with-system-libmpdec / --with-system-expat still uses the vendored headers #98707

Description

Bug report

Your environment

Activity

hroncok commented on Oct 26, 2022

hroncok commented on Oct 26, 2022

hroncok commented on Oct 26, 2022

hroncok commented on Oct 26, 2022

hroncok commented on Oct 26, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions