Repository navigation
Python 3.11 loses the ability to set PYTHON_DECIMAL_WITH_MACHINE #98557
Description
Activity
Cc. @rhettinger for
decimalmodule expertise.FTR, this was removed in GH-29541 (cc. @tiran as PR author and @mdickinson as PR reviewer)
Reacted by Alex WaygoodWhat reads
PYTHON_DECIMAL_WITH_MACHINE? It's not a define to be passed to C but something that used to change what item is selected from:Lines 3882 to 3891 in 75a6fad
Unless you suggest that I instead pass
LIBMPDEC_CFLAGS=-DCONFIG_64=1 -DASM=1on x86_64 andLIBMPDEC_CFLAGS=-DCONFIG_64=1 -DANSI=1 -DHAVE_UINT128_T=1on arm64? (Assuming this list is never changing.)Reacted by Erlend E. AaslandAh, sorry, you are right. I misread the diff from GH-29541; it is used as an index, not a CFLAG.
Yes, it seems this ability is now lost; there is no way to manually override that
AS_CASE. We'd either have to reintroducePYTHON_DECIMAL_WITH_MACHINE, or provide a newconfigureswitch for this, if we were to reintroduce this.You mentioned Homebrew? Does this prevent Homebrew from building and distributing Python 3.11? Is there a Homebrew ticket for this?
From #98557 (comment):
Unless you suggest that I instead pass
LIBMPDEC_CFLAGS=-DCONFIG_64=1 -DASM=1on x86_64 andLIBMPDEC_CFLAGS=-DCONFIG_64=1 -DANSI=1 -DHAVE_UINT128_T=1on arm64? (Assuming this list is never changing.)Then from the OP:
libmpdecproduces different headers depending on how it was built, which is why the setting is important to be able to override. Without it, the_decimalmodule will fail to compile if the default does not match how systemlibmpdecwas built.I suggest you use
LIBMPDEC_CFLAGSto set these flags depending on how yourlibmpdecwas built. That seems to me to be a more robust approach than lettingconfigurechose flags which might not reflect how yourlibmpdecwas built.ISTM we don't need a way to override the
libmpdec_machineswitch inconfigure.Is there a Homebrew ticket for this?
Not yet considering we don't ship betas, but what we'll probably do for 3.11.0 (and what I've done in a local test rc2 build) is a
s/libmpdec_machine=universal/libmpdec_machine=x64/(uint128on arm64) onconfigure. Not a desirable long-term solution but should be ok for a short-term fix.I suggest you use LIBMPDEC_CFLAGS to set these flags depending on how your libmpdec was built. That seems to me to be a more robust approach than letting configure chose flags which might not reflect how your libmpdec was built.
PYTHON_DECIMAL_WITH_MACHINEdid directly match the option you would pass to libmpdec configure. If you passedMACHINE=x64to libmpdec while building then you would pass the same to Python.It made sense and is what Stefan Krah recommended we did.
Might I propose an alternative way forward here however. I propose looking into not setting any CFLAGS for the system libmpdec and checking
MPD_CONFIG_64andMPD_CONFIG_32in_decimalsource code, given mpdecimal.h should tell you how it was built by setting those.FWIW, this is what the relevent section of mpdecimal.h looks like on a different configurations of system libmpdec:
Universal
/* Mac OS X: support for building universal binaries */ #if defined(MPD_CONFIG_64) || defined(MPD_CONFIG_32) #error "cannot use MPD_CONFIG_64 or MPD_CONFIG_32 with universal header." #endif #if defined(CONFIG_64) || defined(CONFIG_32) #error "cannot use CONFIG_64 or CONFIG_32 with universal header." #endif #if defined(__ppc__) #define MPD_CONFIG_32 1 #ifdef UNIVERSAL #define CONFIG_32 #define ANSI #endif #elif defined(__ppc64__) #define MPD_CONFIG_64 1 #ifdef UNIVERSAL #define CONFIG_64 #define ANSI #endif #elif defined(__i386__) #define MPD_CONFIG_32 1 #ifdef UNIVERSAL #define CONFIG_32 #define ANSI #endif #elif defined(__x86_64__) #define MPD_CONFIG_64 1 #ifdef UNIVERSAL #define CONFIG_64 #define ASM #endif #elif defined(__arm64__) #define MPD_CONFIG_64 1 #ifdef UNIVERSAL #define CONFIG_64 #define ANSI #endif #else #error "unknown architecture for universal build." #endifx64 and uint128
/* ABI: 64-bit */ #define MPD_CONFIG_64 1 #ifdef MPD_CONFIG_32 #error "cannot use MPD_CONFIG_32 with 64-bit header." #endif #ifdef CONFIG_32 #error "cannot use CONFIG_32 with 64-bit header." #endifReacted by Erlend E. AaslandMight I propose an alternative way forward here however. I propose looking into not setting any CFLAGS for the system libmpdec and checking
MPD_CONFIG_64andMPD_CONFIG_32in_decimalsource code, given mpdecimal.h should tell you how it was built by setting those.That sounds like a better way forward indeed. I've already pinged Christian on this issue; let's wait and see if he also agrees.
It's been a while here. Can we find a way to move this forward?
It's been a while here. Can we find a way to move this forward?
For 3.11? Unfortunately not; 3.11 only received security fixes. For 3.12 and newer, the environment variables
LIBMPDEC_CFLAGSandLIBMPDEC_LIBSare available if you want to customise how_decimalis built.See also #115119.
Looks like #115406 might have fixed this by moving the assumptions into a
AS_VAR_IF([with_system_libmpdec], [no]in 3.13. Specifically, the flag anddecimal.cchanges rather than the actualpkg-configpart, so the fix works for pre-4.0 too.For 3.11 and 3.12 it was previously unfixable without patching the
configurefile (LIBMPDEC_CFLAGSwon't override it), unless a backport of that new patch is applied.Right, so my gut feel would be that a backport of #115406 probably won't happen; such changes to the
configurescript are always slight behavioural changes to the build system, so backporting them may create havoc for distros.
Bug report
Python 3.10 had the ability to set
PYTHON_DECIMAL_WITH_MACHINEto override the choice of configuration for the_decimalmodule:cpython/setup.py
Lines 2388 to 2397 in dcb342b
Since Python 3.11, this is no longer possible. This feature was necessary, on macOS particularly, with the
--with-system-libmpdecoption if that system libmpdec is configured differently to the default Python config. On macOS, the default Python config forces universal, while settingPYTHON_DECIMAL_WITH_MACHINEallowed it to be single-arch.libmpdecproduces different headers depending on how it was built, which is why the setting is important to be able to override. Without it, the_decimalmodule will fail to compile if the default does not match how systemlibmpdecwas built.Homebrew's Python currently depends on this feature.
A test within CPython also seems to depend on this feature:
cpython/Modules/_decimal/tests/runall-memorydebugger.sh
Lines 63 to 79 in f4c0348
Your environment