Repository navigation
Revert to default fullLTO on Clang #130048
Description
Feature or enhancement
Proposal:
Python 3.12 changed the default for Clang to ThinLTO. However, many people were unaware of this change and did not update their build scripts. This leaves a lot of perf on the table for macOS and possibly some other platforms.
- CPython was bitten by this: Python3.13 performance Issue with python.org macOS installers on ARM Macs #122580
- Faster CPython was bitten by this: clang builds not using full LTO faster-cpython/bench_runner#342
- python-build-standalone (Astral) also seems to not have the fix: On Clang, LTO is ThinLTO, which leaves a lot of performance out for macOS astral-sh/python-build-standalone#528
- msys2/mingw also seems to have not noticed: python: build with full lto msys2/MINGW-packages#23384
This seems to be confusing and tripping up a lot of people. I propose we change the --with-lto default back to full.
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
Activity
brandtbucher commentedon Feb 12, 2025
Heh, this was on my backlog of issues to open. I fully support this... to me,
--with-ltohas always meant "take a long time to compile the fastest executable".corona10 commentedon Feb 13, 2025
Okay, but when the people can be ready to change if we want to change the policy? Why not propose to people to use lto=full if they want? (Yeah It's grumpy comment but I just ask)
corona10 commentedon Feb 13, 2025
Or criteria will be any platform(at least tier1)should not be slow down if we want to change the default build policy?
It means that we would have same criteria for other build configuration too.
Fidget-Spinner commentedon Feb 13, 2025
I think the problem is that people assume lto means lto full, and they forget that it's lto thin now on clang.
Yeah that seems reasonable.
gh-130048: Reintroduce full LTO as default on Clang (GH-130049)
Fidget-Spinner commentedon Feb 13, 2025
For the record, @zanieb tested this on Clang 19.1 (not apple Clang), and couldn't reproduce the slowdowns seen on macOS. So I suspect the latest clang versions have fixed this bug astral-sh/python-build-standalone#529
Revert "pythongh-130048: Reintroduce full LTO as default on Clang (py…
Fidget-Spinner commentedon Feb 13, 2025
I deem it too disruptive to change a configure option twice and flip flop between the defaults just based on what compiler is available. I am thus reverting the change. Sorry!
Revert "gh-130048: Reintroduce full LTO as default on Clang (GH-130049)…
brandtbucher commentedon Feb 13, 2025
Wait, why was this reverted? Full LTO should always be the fastest option, right?
corona10 commentedon Feb 13, 2025
http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html
"In a few cases ThinLTO even outperforms full LTO, most likely because the higher scalability of ThinLTO allows using a more aggressive backend optimization pipeline (similar to that of a non-LTO build)."
So not that always truth. It’s where to bet, but I believe that compiler people make parallel LTO performance be better for the performance stuff too. Basically it’s about the execution mechanism not about the optimization policy.
zanieb commentedon Feb 13, 2025
My benchmarks, though not rigorous, showed full LTO to be a little slower astral-sh/python-build-standalone#529 (comment) 🤷♀ I could do more benchmarks if people want? Or explore the difference here instead of downstream in
python-build-standalonewhere there may be confounding factors?