Repository navigation
sys.executable is sometimes wrong #124241
Description
Activity
I can confirm first behaviour:
sk@note:~/src/peps $ which python sk@note:~/src/peps $ which python3 /usr/local/bin/python3 sk@note:~/src/peps $ bash -c 'exec -a whatever python3 -c "import sys;print(sys.executable)"'But not the second:
$ bash -c 'exec -a /usr/local/bin/python3 python3 -c "import sys;print(sys.executable)"' /usr/local/bin/python3Could you, please, show us output of
which python?Anyway, this seems to be documented: "If Python is unable to retrieve the real path to its executable, sys.executable will be an empty string or None."
Meanwhile, I'll relabel your issue as a feature request.
@skirpichev, maybe this is a permissions issue? I'm on a root user, and I was able to reproduce the second behavior. Running the following on my results in
/usr/bin/echobeing printed:$ bash -c 'exec -a echo python3 -c "import sys;print(sys.executable)"' /usr/bin/echoInterestingly, running it with a path that doesn't exist results in a fatal error:
$ bash -c 'exec -a evil python3 -c "import sys;print(sys.executable)"' Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' Current thread 0x00007f8899f14080 (most recent call first): <no Python frame>Injection of arbitrary files into
sys.executableseems like a security problem to me (but maybe not if this only occurs on root users).maybe this is a permissions issue?
Maybe PATH settings?
That does work for me and doesn't look right:
$ bash -c 'exec -a /usr/bin/evil python3 -c "import sys;print(sys.executable)"' /usr/bin/evilEdit:
Perhaps, this should mentioned somehow in docs.
Injection of arbitrary files into sys.executable seems like a security problem
I doubt you can do too much with this mechanism. Unless there are other options to override arg0. There are e.g. various exec*() calls in POSIX, but it's essentially same. Yes, you can abuse mechanism for yourself (regardless on your rights), but what you gain? I doubt this will be a security issue without decent part of social engineering;)
Reacted by Petr ViktorinI could see it being a problem for applications that use
subprocess.blah_blah([sys.executable, "..."))It's not a terrible vulnerability, but e.g. a user on a Linux machine owned by their company could use it to mess with whatever software the employer has installed to their computer. (Kind of a bad example, but you get the general idea.)
But what evil you could do with this, assuming you can point sys.executable to an arbitrary file? Company software will execute one?
Wait, but that means that someone else put in system some file, that you already can run (suitable permissions, including executable bit). Then why not run directly?
Possibly, I'm just speculating 😃
Regardless, this is a bug. I think we should try to stop
sys.executablefrom holding the wrong thing, rather than just documenting it.I think we should try to stop sys.executable from holding the wrong thing
I don't know universal ways to do so. E.g. on Linux (and I think BSD nowadays), you can use /proc//exe symlink.
I don't know universal ways to do so.
Me neither. But I'm not too sure who to CC on this. I guess if this only affects Linux then
/proc/exeshould work.I guess if this only affects Linux
No, I think it should affect any system with POSIX exec*() functions, for example.
I tried to solve the problem myself, but I am now convinced that it is impossible. Let me explain.
sys.executablecontains the executable before symlinks have been resolved. Unfortunately this is information is stored nowhere, so whatgetpath.pydoes is to guess what the shell probably did to get fromargv[0]to the path of the executable. Unfortunately even good olebashhas some subtleties which are not reflected ingetpath.py. This can lead to a python interpreter using another interpreters standard library.Now one could argue: who cares about the path before symlink resolution, as long as
sys.executableis pointing to the right file, all is OK, just set it to what/proc/self/exepoints to, and we are done.Unfortunately,
venvcares. The python interpreter looks forpyenv.cfgat the place where the symlink is. This has interesting consequences, e.g. if you create a symlink to the interpreter of avenv, or to its containing directory from elsewhere, it will suddenly behave like the original, non-venv interpreter again.I personally don't like that python interpreters behave different if they are a symlink, IMHO a symlink should just be transparent.
venvs could just use a stub file to refer to the original interpreter, using a file with a content like#!/path/to/original/python -Ywhere
-Ywould be a new command line option with the meaning "use this file for calculating the prefix for libraries".I have stumbled upon this problem with my AppImage. I tried to reduce the number of unnecessarily open processes by calling the last command in the AppRun shell script with exec. That last command is Python, i.e., exec python3 .... The output is something like this:
sys.executable: /projects/ratarmount/worktrees/1/ratarmount-x86_64.AppImage.I would have thought that changing
exec "${PYTHON3[0]}" "${ARGS[@]}"toexec -a "${PYTHON3[0]}" "${PYTHON3[0]}" "${ARGS[@]}"would have helped, but it changes nothing.sys.executablestill points to the AppImage. Furthermore, the-aoption is only bash 4.2+ anyway, not POSIX.Trying with
exec -a python3 "${PYTHON3[0]}" "${ARGS[@]}"gives me:./ratarmount-x86_64.AppImage Could not find platform independent libraries <prefix> Could not find platform dependent libraries <exec_prefix> Python path configuration: PYTHONHOME = (not set) PYTHONPATH = (not set) program name = 'python3' isolated = 1 environment = 0 user site = 0 safe_path = 1 import site = 1 is in build tree = 0 stdlib dir = '/opt/_internal/cpython-3.12.11/lib/python3.12' sys._base_executable = '/tmp/.mount_ratarmblPObH/usr/bin/python3' sys.base_prefix = '/opt/_internal/cpython-3.12.11' sys.base_exec_prefix = '/opt/_internal/cpython-3.12.11' sys.platlibdir = 'lib' sys.executable = '/tmp/.mount_ratarmblPObH/usr/bin/python3' sys.prefix = '/opt/_internal/cpython-3.12.11' sys.exec_prefix = '/opt/_internal/cpython-3.12.11' sys.path = [ '/opt/_internal/cpython-3.12.11/lib/python312.zip', '/opt/_internal/cpython-3.12.11/lib/python3.12', '/opt/_internal/cpython-3.12.11/lib/python3.12/lib-dynload', ] Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' Current thread 0x0000754a239a7740 (most recent call first): <no Python frame>
Exporting
PYTHONHOMEdoes not work because-I(isolated mode) is used in the AppImage.My bad, in my case, the "problem" was a patch by python-appimage inside
encodings/__init__.py:command = env["APPIMAGE_COMMAND"].On most Linux systems, this could be solved by reading the symlink target of
/proc/self/exe.
Bug report
Bug description:
On Linux,
sys.executableis determined usingargv[0], and trying to find that in thePATH. This presumes that the Python interpreter has been started by a shell that interpretsPATHin the canonical way. There is no guarantee that this happens at all, sosys.executablemay point to something completely different, or is even the empty string.As an example:
should output the path of the Python executable, but simply outputs nothing.
Even worse, one can make Python output a wrong executable, as in
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
/proc/self/exeto determinesys.executable#145486