Repository navigation
[BOLT] [3.12] Python 3.12.7 --enable bolt option not working #124948
Description
Bug report
Bug description:
Hi together,
After updating llvm-bolt to 19.1.0 it is not possible to use the --enable-bolt function anymore.
Following can be found in the log:
BOLT-INFO: 0 out of 5950 functions in the binary (0.0%) have non-empty execution profile
BOLT-INSTRUMENTER: Number of indirect call site descriptors: 7119
BOLT-INSTRUMENTER: Number of indirect call target descriptors: 5874
BOLT-INSTRUMENTER: Number of function descriptors: 5874
BOLT-INSTRUMENTER: Number of branch counters: 78400
BOLT-INSTRUMENTER: Number of ST leaf node counters: 40936
BOLT-INSTRUMENTER: Number of direct call counters: 0
BOLT-INSTRUMENTER: Total number of counters: 119336
BOLT-INSTRUMENTER: Total size of counters: 954688 bytes (static alloc memory)
BOLT-INSTRUMENTER: Total size of string table emitted: 133862 bytes in file
BOLT-INSTRUMENTER: Total size of descriptors: 7642344 bytes in file
BOLT-INSTRUMENTER: Profile will be saved to file /tmp/pkg/src/Python-3.12.7/libpython3.12.so.1.0.bolt
BOLT-INFO: 65850 instructions were shortened
BOLT-INFO: removed 134 empty blocks
BOLT-INFO: UCE removed 1155 blocks and 71042 bytes of code
BOLT-INFO: padding code to 0xe00000 to accommodate hot text
BOLT-INFO: output linked against instrumentation runtime library, lib entry point is 0xf90930
BOLT-INFO: clear procedure is 0xf8c380
BOLT-INFO: patched build-id (flipped last bit)
BOLT-INFO: setting __bolt_runtime_start to 0xf908e0
BOLT-INFO: setting __bolt_runtime_fini to 0xf90930
BOLT-INFO: setting __hot_start to 0x800000
BOLT-INFO: setting __hot_end to 0xdf7438
BOLT-ERROR: unable to get new address corresponding to input address 0x1a6e11 in function sre_ucs1_match/1. Consider adding this function to --skip-funcs=...
make[1]: *** [Makefile:856: profile-bolt-stamp] Error 1
make[1]: Leaving directory '/tmp/pkg/src/Python-3.12.7'
make: *** [Makefile:885: bolt-opt] Error 2
==> ERROR: A failure occurred in build().
so, it appears to fail at the instrumentation and suggest to add --skip-funcs=sre_ucs1_match/*.
CPython versions tested on:
3.12
Operating systems tested on:
Linux
Activity
Eclips4 commentedon Oct 3, 2024
cc @corona10
ZIZUN commentedon Nov 2, 2024
I will take a look
ZIZUN commentedon Nov 3, 2024
I tested this issue on my setup with CPython 3.12 and LLVM-BOLT version 19.1.0 on a Ubuntu(20.04) environment, and I did not encounter any problems. The --enable-bolt function worked as expected. It appears that this issue might be other environment problems. @ptr1337
ptr1337 commentedon Dec 23, 2024
After Python 3.13 got pushed to archlinux stable, im still not able to bolt it with the same issues.
Since the llvm-bolt package is not in the archlinux repository, you can test it with following docker container:
ms178 commentedon Dec 23, 2024
I can confirm this issue using Clang-20git (c660b281b60085cbe40d73d692badd43d7708d20) on CachyOS with Python 3.13.1:
And if I do as advised by BOLT, I see another function that errors out instead:
BOLT-ERROR: unable to get new address corresponding to input address 0x4fa143 in function sre_ucs1_match/1(*2). Consider adding this function to --skip-funcs=...ptr1337 commentedon Dec 27, 2024
Also, another idea:
Instead of relaying on instrumentation (which works on most enviroments) we could also add an option.
@lseman provided an option a while ago at following patch: https://termbin.com/zksh
ptr1337 commentedon Dec 27, 2024
Adding to the bolt commands:
--skip-funcs=sre_ucs1_match/1,_PyEval_EvalFrameDefault.localalias/1appears to fix it.
indygreg commentedon Dec 31, 2024
The
unable to get new address corresponding to input addresserror was added in LLVM 19.1 by llvm/llvm-project#89681. It can trigger when using computed gotos in PIC compiled code. Not sure about the original report, but_PyEval_EvalFrameDefaultuses computed gotos, making it BOLT incompatible with LLVM 19.1. (And behavior on older LLVM versions may be incorrect, leading to buggy behavior.)I think a proper fix here is to add any functions with computed gotos to the BOLT exclusion list. Presumably a future LLVM release will gain the ability to perform these dynamic relocations, so we'll [eventually] want some form of LLVM version sniffing to control the behavior.
ms178 commentedon Dec 31, 2024
@indygreg This might be the relevant LLVM-MR for gaining that functionality: llvm/llvm-project#120267
indygreg commentedon Jan 1, 2025
Nice find, @ms178! I agree that PR looks promising. Hopefully it makes LLVM 20.
While I'm here, python-build-standalone is working around the issue with a patch at astral-sh/python-build-standalone#463. I needed to add
-skip-funcs=_PyEval_EvalFrameDefault,sre_ucs1_match/1,sre_ucs2_match/1,sre_ucs4_match/1to bothBOLT_INSTRUMENT_FLAGSandBOLT_APPLY_FLAGSto get things to work.ms178 commentedon Jan 1, 2025
@indygreg While at it, I'd suggest to modernize the BOLT flags a bit:
-reorder-functions=cdsortand-split-strategy=cdsplitare now the state-of-the art according to a recent LLVM presentation from the BOLT devs (https://llvm.org/devmtg/2024-03/slides/practical-use-of-bolt.pdf - slide 33 onwards). That would need a bit of testing though.indygreg commentedon Jan 1, 2025
Good idea!
FWIW
-reorder-functions=hfsort+is transparently rewritten tocdsort:But
-split-strategydefaults toprofile2.liusy58 commentedon Jan 3, 2025
strategy=cdsplitis only fully supported in X86 but not on AArch64, but I am now working on it recently~zanieb commentedon Jan 4, 2025
It's a bit off-topic here, but could you expand on this? Would it fail at build or runtime on aarch64?
liusy58 commentedon Jan 6, 2025
Is there something like Discord for CPython? Maybe it is more convenient we can chat there ?
zanieb commentedon Jan 6, 2025
My Discord username is
zanieb— you're welcome to reach out there. I'm on the Astral Discord and the Python Discord (though I'm not sure what channel would be used in the latter). We can also talk in #128514 which is dedicated to this topic — that may be best for visibility.liusy58 commentedon Jan 6, 2025
Ok, it seems that there is no notification in github. So I often miss important message...