Repository navigation
Windows python3 executable #99185
Description
Activity
@zooba do you have any proposals for this? On unix, we have symlinks up to the executable with the minor version:
python>python3>python3.12. I assume this is not the case on Windows because symlinks aren't reliably supported, so it would require us to copy the interpreter. Do you know if there is any other mechanism that allow us to implement this reliably? If not, would it be reasonable to install very simple launcher executables that mimic the symlink behavior? I definitely think this is a goal worth pursuing.One option is simply to add a copy of
python.exenamedpython3.exein the base install directory. This appears to be how PIP is handled, withpip.exe,pip3.exeandpip3.X.exeall being copies of the same executable located in a Scripts subdirectory.pip's case is a bit different, all the executables are launchers for a Python module.
a user following almost any online tutorial, copy/pasting installation scripts, etc. will fail because all are written for *nix platforms that standardize on
python3as the name of the executable.Hmm, I checked about 10 of the top Google results for
python tutorial, and that's not at all what I saw...Thanks for the reply and attention.
pip's case is a bit different, all the executables are launchers for a Python module.
True, though I believe on Windows, python.exe simply invokes functions in python3X.dll. Here's a directory listing:
08/30/2021 07:36 PM 101,608 python.exe 08/30/2021 07:36 PM 59,624 python3.dll 08/30/2021 07:36 PM 4,488,424 python39.dll 08/30/2021 07:36 PM 2,504,187 python39.zipThe executable stubs are even smaller than the pip ones:
11/03/2021 08:44 AM 106,337 pip.exe 11/03/2021 08:44 AM 106,337 pip3.9.exe 11/03/2021 08:44 AM 106,337 pip3.exeYes, symbolic links exist on Windows, but only for NTFS, not FAT32. IMO, perhaps just following pip's lead and copying the executable stubs might be the lowest friction solution.
Hmm, I checked about 10 of the top Google results for python tutorial, and that's not at all what I saw...
Well, the first result...python.org...currently uses
python3.11. How about I sidestep this and just modify my original assertion and say that "much of the documentation I send my students to" usespython3. But even if it usespython, my Mac users will have issues.The REAL problem is that there is inconsistency between platforms and documentation.
pythonworks by default on Windows butpython3orpython3.xdoes not.pythondoes not work by default on MacOS or Linux (Debian/Ubuntu anyway), butpython3andpython3.xdoes. I sent students to the MariaDB and MySQL connector pages. Windows folks had issues runningpython3from MariaDB docs and Mac users had issues withpythonin the MySQL docs.Likewise,
pipworks by default on Windows but not on Mac/*nix, which needpip3; but many package installation guides give command line examples usingpipand notpip3. And then there's thepylauncher for Windows, which is cool but doesn't exist on Mac/*nix and is unfortunately referenced in a number of tutorials. And I think the/usr/bin/pythonsymbolic link is apparently supposed to be managed usingupdate-alternativeson Linux, which I guess is similar, but is not installed by default nor is it available on Mac.Anyway, point is there is inconsistency across platforms. And while some inconsistency is unavoidable, these seem unnecessary and do cause confusion for students...and distress for teachers. If we can just get
python,python3,python3.X,pip,pip3, andpip3.xall working by default across all major platforms, that would be fantastic. Short of this, at least the_3and_3.xones......I guess asking for a default
/usr/bin/pythonand/usr/bin/piplinks to be added for Mac/*nix is probably out of scope for the issue, huh? ;)100MB
You mean 100 kB.
The python.org tutorial explains what to use for different systems, showing
python3.11,pyandpython. Does your own tutorial also do that (in the intro chapter, not just in the FAQ) but students just choose to ignore that? (Maybe include a note that asking about it will cost them points in their next exam :-)Pedantry aside, yes my CS majors must of course learn the intricacies, though you'd be surprised at how little motivation many show to learn even the basics of terminal operations. I can and will hold these students accountable as it is a learning outcome. HOWEVER, students in other majors take classes or are simply given ancillary assignments in Python to provide exposure and understanding (and recruiting), not to master it. They copy/paste blocks of code that they find online or that I write. Having to constantly provide "translations" and then having them still encounter roadblocks and frustrations as they have to seek and wait potentially hours for assistance due to the need to add or remove a '3'...is avoidable pain. Pain that can needlessly drive away potentially talented technology majors, which is counter to our mission.
This is adding little to this issue.
java/javac,dotnet,node/npm,gcc,clang,go, etc., etc. all are CLIs that are at least semantically identical cross-platform. There seems little reason for Python to be the outlier, especially as it is billed as an approachable introduction to coding.True, though I believe on Windows, python.exe simply invokes functions in python3X.dll. Here's a directory listing:
AFAIK it parses the command line and envvars and invokes the interpreter via
python3X.dll.And I think the
/usr/bin/pythonsymbolic link is apparently supposed to be managed usingupdate-alternativeson Linux, which I guess is similar, but is not installed by default nor is it available on Mac.This is actually a Debian (and naturally Ubuntu and anything else Debian-based) specific thing btw, other distributions don't have that.
Yes, symbolic links exist on Windows, but only for NTFS, not FAT32.
And IIRC there's a toggle setting to enable support for them 🙃
IMO, perhaps just following pip's lead and copying the executable stubs might be the lowest friction solution.
Well, pip's executable stubs (aka packaging entrypoint launchers) are a bit different, it's not as straight-forward here. Like I said above, I am pretty sure the executable is more than just a stub, so we need to create stubs.
...I guess asking for a default
/usr/bin/pythonand/usr/bin/piplinks to be added for Mac/*nix is probably out of scope for the issue, huh? ;)No, if we are fixing the inconsistencies, I think it should be done for all platforms. On macOS it can be done via symlinks though, I am actually surprised you mentioned it doesn't already do this.
This is adding little to this issue.
java/javac,dotnet,node/npm,gcc,clang,go, etc., etc. all are CLIs that are at least semantically identical cross-platform. There seems little reason for Python to be the outlier, especially as it is billed as an approachable introduction to coding.I agree, it's unfortunate. If you are teaching introduction to Python, I would consider using IPython or Jupyter though. Anyway, I am glad to see you are advocating for your students, I wish my teachers did that when I was in college 😅
Thanks for the clarifications, @FFY00!
On macOS it can be done via symlinks though, I am actually surprised you mentioned it doesn't already do this.
Surprised me, too. I just installed the latest from python.org just to make sure:
% where python python not found % where python3 /Library/Frameworks/Python.framework/Versions/3.11/bin/python3 /Library/Frameworks/Python.framework/Versions/3.10/bin/python3 /usr/local/bin/python3 /usr/bin/python3 % ls -l /usr/local/bin/python* lrwxr-xr-x 1 root wheel 70 Dec 16 14:48 /usr/local/bin/python3 -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3 lrwxr-xr-x 1 root wheel 77 Dec 16 14:48 /usr/local/bin/python3-config -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3-config lrwxr-xr-x 1 root wheel 78 Dec 16 14:48 /usr/local/bin/python3-intel64 -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3-intel64 lrwxr-xr-x 1 root wheel 73 Sep 23 12:19 /usr/local/bin/python3.10 -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10 lrwxr-xr-x 1 root wheel 80 Sep 23 12:19 /usr/local/bin/python3.10-config -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10-config lrwxr-xr-x 1 root wheel 81 Sep 23 12:19 /usr/local/bin/python3.10-intel64 -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10-intel64 lrwxr-xr-x 1 root wheel 73 Dec 16 14:48 /usr/local/bin/python3.11 -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11 lrwxr-xr-x 1 root wheel 80 Dec 16 14:48 /usr/local/bin/python3.11-config -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11-config lrwxr-xr-x 1 root wheel 81 Dec 16 14:48 /usr/local/bin/python3.11-intel64 -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11-intel64 % ls -l /usr/bin/python* -rwxr-xr-x 76 root wheel 167120 Aug 24 04:59 /usr/bin/python3Would also be nice to have it do the same for pip as there's only a /usr/local/bin/pip3 and pip3.x.
Debian/Ubuntu .deb package seems also not to add a /usr/bin/python or /usr/local/bin/python symbolic link. python3-pip DOES however have pip, pip3 and pip3.x script replicas. Different teams making the installers, I guess.
FWIW, I'll go to my grave not understanding the real distinction between /usr/bin and /usr/local/bin. 😆
If you are teaching introduction to Python, I would consider using IPython or Jupyter though.
Agreed!! We include Python in a (very) intro course, a "scripting and automation" course, database, and data science. Each kinda lends itself to different environments. Intro uses some kind of simple IDE (currently IDLE maybe?). Automation I think uses Mu, Data Science uses Jupyter. And my latest pain is in Database, where I need them to "experience" accessing a database from an application...here I use VSCode and CLI since they are connecting to a local MariaDB daemon.
I guess the challenge of having such a powerful, general purpose scripting language is that it is used in so many different ways...
I've been hesitant to add more executables to the Python install directory, because enough users already have messed up PATH that adding more is only going to be a bigger problem.
Any changes at all are going to cause churn, which is going to break people, which is going to use up any remaining goodwill we have left. I'd rather save the churn until we actually have a good, long-term fix. But I also don't know what that fix will be.
My best guess right now is that all the scenarios that would benefit from having "python3" be slightly more consistent (it's still not going to behave the same) would benefit far more from using a distro (who can easily do this themselves) or a preinstalled/configured environment such as via Jupyter or Minecraft. The python.org install of Python is such a long way from being a beginner friendly option that I'd rather leave that space clearly open for someone who wants to focus on that audience.
The Store install is a significant concession, but also properly handles the PATH modifications so I'm far less concerned about multiple executables (and the stub that launches the store was added by Microsoft in support - we don't control that from here, though I personally have the contacts to make changes if needed). That said, if they don't fix up the execution alias UI some time soon I'll seriously consider dropping some of the entry points.
enough users already have messed up PATH that adding more is only going to be a bigger problem.
To offer a really concrete example here:
- User installs (hypothetical) 3.11.2 with added python3.exe. 3.11 gets pushed to the front of PATH
- User installs security update to 3.10. 3.10 gets pushed to the front of PATH
- Now
python.exeis 3.10 andpython3.exeis 3.11. The only way to fix it is to manually modify PATH, and a user who knows how to do that could've set it up manually in a way that wouldn't break like this.
I understand, and agree with your point regarding churn, however if we go that route I think we should decide what is our stance regarding installing versioned executable in general and document it, along with whatever exceptions we might have (eg. Windows if we decide that we want to install versioned executables as a general practice, or Linux if we decide we don't).
Having somewhere in the documentation where we can point people to learn what exactly is the difference in behavior between the different platforms would go a long way to mitigate this issue.
- Now
python.exeis 3.10 andpython3.exeis 3.11.
and
pip3.exepresumably would be 3.10. Yeah. That is a mess. Yuck!As a "convenience" maybe coming up with a managed "standard" method of setting the current version that is consistent across platforms? Perhaps pysetver or something. It could present a list of known installs (in the path on Windows and common install locations elsewhere) and allow you to pick one to set as active? Would modify the path on Windows and update the symlink elsewhere? Kinda sorta like venv, but for the base install? Dunno. Might just be more churn...
Having somewhere in the documentation where we can point people to learn what exactly is the difference in behavior between the different platforms would go a long way to mitigate this issue.
Doesn't fix the copy/paste issue, but I agree that it is a good middle ground if no universal fix can be envisioned.
Reacted by Steve Dower- Now
As a "convenience" maybe coming up with a managed "standard" method of setting the current version that is consistent across platforms? Perhaps pysetver or something.
This would be great, but unfortunately doesn't work well (on Windows at least). A process can't change the environment of its parent process, which would mean it would have to be a batch file and a PowerShell script and a Bash script to handle the three most popular shells (on Windows - Git Bash gets a lot of use), and nobody has signed up for that.
Would modify the path on Windows and update the symlink elsewhere
This is actually my ideal design, but it requires OS changes to work properly. I'm trying to argue those into existence before I give up and look for an alternative (potentially renaming the
py.exelauncher topython.exeandpython3.exe... which it already supports, but obviously the default counts for a lot... and I already know how this will break some users).A non-terminal interface such as Jupyter, Mu or VS Code, is always going to avoid this level of confusion. It's a shame, but also pretty irreversible, that terminal skills are a prerequisite to using Python.
A process can't change the environment of its parent process, which would mean it would have to be a batch file and a PowerShell script and a Bash script to handle the three most popular shells (on Windows - Git Bash gets a lot of use), and nobody has signed up for that.
True...but...to change the path permanently, you're modifying the registry using setx or similar. Which will require closing and reopening the command prompt anyway. So using C/Win32 or python + the winreg module could do this. You'd just need to print something like "Changes won't be effective until you close and reopen the command prompt" at the end.
Would be happy to throw together some proof-of-concept code for this if interested.
But, yes, a more elegant "work immediately" option would need to BOTH modify the registry and refresh the current environment from the registry. Sadly, an age-old issue since...ages old. Like me. :D And one that would indeed require a bat/ps1/sh script.
potentially renaming the
py.exelauncher topython.exeandpython3.exeInteresting. Would be the "most similar" to symlinks on other platforms. But I can imagine this would almost have to be an "opt in" option. At least if other Python installs are detected.
No perfect solutions here, huh? At least without a Tardis.
Reacted by Steve Dowermaybe python can ship with a "manage with pyenv" as the standard installation, so users who want some facility with managing versions and the whole python3, 3.X executable thing can have that managed for them, and we can decouple the "manage python version stuff" repo from "cpython". be nice if cpython wasn't even in the business of managing the active version, paths, etc.
Reacted by Steve Dower24 remaining items
How does this work with virtual environments?
See the searching for interpreters guidance for details, but tl;dr, it has virtualenv support.
at which point it could be a drop in for the Python Launcher for Windows and possibly distributed with CPython
Afraid not, Brett's launcher has about 1% of the functionality of our current one, and he has no desire to add the rest (nor would I recommend adding the rest - shebang emulation turns out to be nearly impossible to get right). It could certainly be distributed on other platforms though.
Reacted by Jason R. CoombsThe fact that virtual environments don't create a
python3executable leads to very confusing behavior:C:\Users\johnsmith>python3 -c "import sys; print(sys.executable)" C:\Users\johnsmith\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\python.exe C:\Users\johnsmith>python3 -m venv test_venv C:\Users\johnsmith>.\test_venv\Scripts\activate.bat (test_venv) C:\Users\johnsmith>python3 -c "import sys; print(sys.executable)" C:\Users\johnsmith\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\python.exe (test_venv) C:\Users\johnsmith>python -c "import sys; print(sys.executable)" C:\Users\johnsmith\test_venv\Scripts\python.exeSo even if the user has installed Python "correctly" (i.e. through the Microsoft Store), they can still run into weird inconsistencies.
Please see PEP 773 and show your support for it on the Discourse thread (linked from the PEP). Until this is either accepted or rejected, we won't be changing anything else about how Python installs on Windows, and if it's rejected then it's also unclear what will be changed.
The proposed behaviour in PEP 773 includes a
python3command that will run your active virtual environment, which should solve your confusion.With the new installer, the recommended command to launch the default Python on Windows will be
pythonNot off to a great start. As long as it at least works with
python3that's definitely an improvement though.The best option for users at this point is clearly to use uv.
Not off to a great start.
The recommended command everywhere is
python, and has been for years. It's no good blaming upstream (us) for downstream (distro) changes to our recommendations.The best option for users at this point is clearly to use uv.
They're welcome to, we aren't forcing anyone to use our builds. We can't officially recommend their builds, though, because we can't take responsibility for their security or the modifications they apply before and during building.
The recommended command everywhere is
python, and has been for years. It's no good blaming upstream (us) for downstream (distro) changes to our recommendations.
https://docs.python.org/3/using/unix.html#python-related-paths-and-filesCould you clarify what is meant here?
It's out of date, would you like to fix it?
Also see https://peps.python.org/pep-0394/ and particularly this section, and be aware that this is quite a difficult PEP to read and interpret, as it bounces fairly freely between intention, expectation, and recommendation.
The "using" docs also tend towards realistic expectation, rather than specification, as they're supposed to be the place users get sent by anyone who doesn't want to document how to run Python on every single OS out there. As Zach says, it's likely that they're out of date or not very helpful, as they don't get rewritten often.
While far from being universally available,
pythonremains the preferred spelling for explicitly invoking Python, as this is the spelling that virtual environments make consistently available across different platforms and Python installations.If the
pythoncommand is installed, it is expected to invoke either the same version of Python as thepython3command or as thepython2command.I mean... you can see why the official recommendation didn't catch on.
Anyway
uvhas solved this issue (and many many others) so I will bow out of this thread.The installer referred to in this issue has now been removed (#153511), and so this issue is expired. Please use the Python install manager to install Python, and file any issues on the pymanager repo.
ISSUE:
On Windows, a Python 3 install directory contains
python.exe. However, there is nopython3.exe,python3.cmdor similar. Thus, a user following almost any online tutorial, copy/pasting installation scripts, etc. will fail because all are written for *nix platforms that standardize onpython3as the name of the executable.Worse, there is a
python3.exein a defaultAppData\Local\Microsoft\WindowsAppsdirectory which is a stub to open the Windows Store to download Python 3. So the unsuspecting user is greeted with the option to install Python 3 from the Windows Store, which could break an existing installation and certainly will not do what they intended.This is obviously only an issue for novice users. But I teach novice users. And this comes up numerous times each semester despite posting FAQ's and warnings related to this issue. And it seems very easy to fix in the default install.
This seems like it should be an issue long since reported, but I cannot find it by searching. So I apologize if this is a duplicate.
REQUESTED FIX:
One option is simply to add a copy of
python.exenamedpython3.exein the base install directory. This appears to be how PIP is handled, withpip.exe,pip3.exeandpip3.X.exeall being copies of the same executable located in a Scripts subdirectory.Alternatively, adding a
python3.cmdscript that calls python.exe and forwards all args would serve the same function. Though I'm unsure that saving 100MB would be worth the potential confusion.Linked PRs