Repository navigation
Build changes for Windows free-threaded builds #112984
Description
Activity
Deliberately not updating
sys.version,sys.thread_infoor the opening prompt of the REPL in this PR, because those affect all platforms and other people will want to argue about them.So I've added an installer option in this PR
Normal first page:
Where you go after clicking "Customize":
The page with the new option (last checkbox):
It's labelled "Download" because the files will be on python.org and downloaded on demand, so if you don't have internet access you won't get them. The installer already has a
/layoutoption to pre-download all its optional components, so it's nothing new here.After install, there's a
python3.13t.exeandpythonw3.13t.exeinstalled alongside the usualpython.exe/pythonw.exe. Right now, they're already registered as normal, which meanspy.exe -3.13twill select it whilepy.exe -3.13gets the usual version. There's also a "Python 3.13 (64-bit, free-threaded)" shortcut added to the start menu.Unfortunately,
py.exe -3will select the free-threaded one, which I don't like but don't really have a great way to solve it - the easiest way is to make it so you need something likepy.exe -V:PythonFreeThreaded/3.13to get the free-threaded build, which has the advantage thatpy.exe -V:PythonFreeThreaded/will get the latest free-threaded build available, but a disadvantage is that shebang parsing won't be able to select it (without writing special case code in the launcher, which I'd rather not do if it can be avoided). On balance, I think it's probably best to change the company name toPythonFreeThreadedor something like that - open to suggestions!Right now, venv will create
python3.13t.exein the new environment. I'm tempted to say it should createpython.exeinstead, as that will make it easier to use. It probably requires more significant changes to venv though, so I'd rather do it later. Differentvenvlauncherimplementations are needed because they "know" the name of the real executable will be, so we already do a sneaky rename. (Possibly that's being done wrong in the current PR...)I haven't yet created a Nuget package or embeddable distro for the files. I'm leaning toward
python-freethreadedandpythonarm64-freethreadedpackages on Nuget. This will really help out people running on CI, as they'll be able to easily get a set of binaries independent from what the CI image contains and without having to run an installer as part of their job. We could also just put a zip file of the binaries on python.org, but I fear it would clutter the list of downloads (which is already considered cluttered). No separate embeddable distro at this stage, people can build it easily enough if they want it.Today's updates:
- fixed venv (possibly more broadly than necessary to avoid having too many special cases for this... @vsajip, I'd appreciate a review)
- updated the installer/layout tools
- started release builds PR Implement free-threaded build for Windows release release-tools#81
There's quite a lot of venv-related change - is there any discussion or design documentation anywhere? For example, about the
venvlauncherexecutable - what is the need/rationale for it, as it appears to be new in this PR?The
venvlauncher.cis justlauncher.cboiled down to just the venv parts, and none of the py.exe parts. The only change is that the executable embeds the expected Python executable name at compile time, as we can't just assume it'll bepython[_d].exeanymore.The changes to
venv\__init__.pyshould just streamline things for Windows. We need to more carefully pick the right redirector, and we can't rely on the name beingpython.exeanymore, and trying to handle the symlink->copy fallback per-file was already pretty rough as we aren't copying the regular executables.The design is the same as it was before, it's mainly just refactoring how we get there.
- added a commit that references this issue
on Dec 21, 2023 Do you plan to provide freethreaded debug (
td) binaries?
That does start a combinatorial explosion of builds, but I figure that people will need all the debugger help they can get. (And they'll want to run benchmarks so building onlydtisn't appropriate.)They're included with the installer is the current PR. No option, you just get a
python3.13t_d.exealong with thepython3.13t.exe. If size becomes an issue then they'll be the first thing to go, but I think they only add about 5MB to the installer package (which is only downloaded when the option is selected during install).I'm not convinced the debug binaries are that helpful anyway, compared to just building it yourself. If you're in a position where they are helpful, you almost certainly have the build tools already (or know that you're doing something weird and ought to have the build tools).
Reacted by Petr Viktorin- added a commit that references this issue
on Jan 4, 2024 5 remaining items
@itamaro Yes, nogil tests now need to pass
--disable-giltort.batin order to get the correct executable name. I'm not sure where best to put that, I don't maintain any buildbots.the builder is configured here. is the solution to add
testFlags = ["--disable-gil"]? (I'm guessing these flags are passed down tort.bat?)we'll see if python/buildmaster-config#458 fixes it
- added a commit that references this issue
on Jan 22, 2024 I don't see any need to keep this open, so closing. Feel free to reopen if there's a reason.
Reacted by Sam Gross
We should use a different
$(IntDir)and$(OutDir)for free threaded builds so that it's faster to switch between them and so they can be built in parallel.The output filename should be
python{major}.{minor}t.exeandpython{major}{minor}t.dll. Debug builds arepython{major}.{minor}t_d.exeandpython{major}{minor}t_d.dll.The ABI tag becomes
.cp{major}{minor}t-{plat}All stdlib .pyd files get an ABI tag.
The
python.batis updated as normal (overwriting the last yesgil build).The installer should not include a
python.exe, to avoid causing PATH conflicts, but Nuget, embeddable and Store packages should include a copy/alias.PEP 514 registration (a.k.a.
sys.winver) should use{major}.{minor}t-{plat}.Tests that rely on the executable name should use
sys.executablewherever possible, and hopefully there's a support variable already for the rest.All of these only take effect when
$(DisableGil) == 'true'(which is set bybuild.bat --disable-gil, but the build variable is canonical).Linked PRs