Skip to content

pyrepl: restructure imports to give a meaningful error when curses is missing on POSIX #130046

Closed as duplicate
Closed as duplicate
pyrepl: restructure imports to give a meaningful error when curses is missing on POSIX#130046
Labels
stdlibStandard Library Python modules in the Lib/ directorytopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

Description

@vfazio
vfazio
opened on Feb 12, 2025

Bug report

Bug description:

If CPython is built and deployed onto a POSIX system without curses, we get an error about importing msvcrt which is a Windows library and thus confusing

# python3
random: python3: uninitialized urandom read (24 bytes read)
Python 3.13.2 (main, Feb 12 2025, 13:57:13) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
warning: can't use pyrepl: No module named 'msvcrt'

Here's the import log

# trying /usr/lib/python3.13/_pyrepl/__main__.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/_pyrepl/__main__.abi3.so
# trying /usr/lib/python3.13/_pyrepl/__main__.so
# trying /usr/lib/python3.13/_pyrepl/__main__.py
# trying /usr/lib/python3.13/_pyrepl/__main__.pyc
# code object from '/usr/lib/python3.13/_pyrepl/__main__.pyc'
# trying /usr/lib/python3.13/_pyrepl/main.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/_pyrepl/main.abi3.so
# trying /usr/lib/python3.13/_pyrepl/main.so
# trying /usr/lib/python3.13/_pyrepl/main.py
# trying /usr/lib/python3.13/_pyrepl/main.pyc
# code object from '/usr/lib/python3.13/_pyrepl/main.pyc'
# trying /usr/lib/python3.13/_pyrepl/simple_interact.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/_pyrepl/simple_interact.abi3.so
# trying /usr/lib/python3.13/_pyrepl/simple_interact.so
# trying /usr/lib/python3.13/_pyrepl/simple_interact.py
# trying /usr/lib/python3.13/_pyrepl/simple_interact.pyc
# code object from '/usr/lib/python3.13/_pyrepl/simple_interact.pyc'
# trying /usr/lib/python3.13/_pyrepl/readline.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/_pyrepl/readline.abi3.so
# trying /usr/lib/python3.13/_pyrepl/readline.so
# trying /usr/lib/python3.13/_pyrepl/readline.py
# trying /usr/lib/python3.13/_pyrepl/readline.pyc
# code object from '/usr/lib/python3.13/_pyrepl/readline.pyc'
# trying /usr/lib/python3.13/_pyrepl/unix_console.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/_pyrepl/unix_console.abi3.so
# trying /usr/lib/python3.13/_pyrepl/unix_console.so
# trying /usr/lib/python3.13/_pyrepl/unix_console.py
# trying /usr/lib/python3.13/_pyrepl/unix_console.pyc
# code object from '/usr/lib/python3.13/_pyrepl/unix_console.pyc'
# trying /usr/lib/python3.13/_pyrepl/curses.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/_pyrepl/curses.abi3.so
# trying /usr/lib/python3.13/_pyrepl/curses.so
# trying /usr/lib/python3.13/_pyrepl/curses.py
# trying /usr/lib/python3.13/_pyrepl/curses.pyc
# code object from '/usr/lib/python3.13/_pyrepl/curses.pyc'
# trying /root/_curses.cpython-313-arm-linux-gnueabi.so
# trying /root/_curses.abi3.so
# trying /root/_curses.so
# trying /root/_curses.py
# trying /root/_curses.pyc
# trying /usr/lib/python3.13/_curses.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/_curses.abi3.so
# trying /usr/lib/python3.13/_curses.so
# trying /usr/lib/python3.13/_curses.py
# trying /usr/lib/python3.13/_curses.pyc
# trying /usr/lib/python3.13/lib-dynload/_curses.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/lib-dynload/_curses.abi3.so
# trying /usr/lib/python3.13/lib-dynload/_curses.so
# trying /usr/lib/python3.13/lib-dynload/_curses.py
# trying /usr/lib/python3.13/lib-dynload/_curses.pyc
# trying /usr/lib/python3.13/site-packages/_curses.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/site-packages/_curses.abi3.so
# trying /usr/lib/python3.13/site-packages/_curses.so
# trying /usr/lib/python3.13/site-packages/_curses.py
# trying /usr/lib/python3.13/site-packages/_curses.pyc
# trying /root/curses.cpython-313-arm-linux-gnueabi.so
# trying /root/curses.abi3.so
# trying /root/curses.so
# trying /root/curses.py
# trying /root/curses.pyc
# trying /usr/lib/python3.13/curses.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/curses.abi3.so
# trying /usr/lib/python3.13/curses.so
# trying /usr/lib/python3.13/curses.py
# trying /usr/lib/python3.13/curses.pyc
# trying /usr/lib/python3.13/lib-dynload/curses.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/lib-dynload/curses.abi3.so
# trying /usr/lib/python3.13/lib-dynload/curses.so
# trying /usr/lib/python3.13/lib-dynload/curses.py
# trying /usr/lib/python3.13/lib-dynload/curses.pyc
# trying /usr/lib/python3.13/site-packages/curses.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/site-packages/curses.abi3.so
# trying /usr/lib/python3.13/site-packages/curses.so
# trying /usr/lib/python3.13/site-packages/curses.py
# trying /usr/lib/python3.13/site-packages/curses.pyc
# trying /usr/lib/python3.13/_pyrepl/_minimal_curses.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/_pyrepl/_minimal_curses.abi3.so
# trying /usr/lib/python3.13/_pyrepl/_minimal_curses.so
# trying /usr/lib/python3.13/_pyrepl/_minimal_curses.py
# trying /usr/lib/python3.13/_pyrepl/_minimal_curses.pyc
# code object from '/usr/lib/python3.13/_pyrepl/_minimal_curses.pyc'
# destroy _pyrepl._minimal_curses
# trying /usr/lib/python3.13/_pyrepl/windows_console.cpython-313-arm-linux-gnueabi.so
# trying /usr/lib/python3.13/_pyrepl/windows_console.abi3.so
# trying /usr/lib/python3.13/_pyrepl/windows_console.so
# trying /usr/lib/python3.13/_pyrepl/windows_console.py
# trying /usr/lib/python3.13/_pyrepl/windows_console.pyc
# code object from '/usr/lib/python3.13/_pyrepl/windows_console.pyc'
# trying /root/msvcrt.cpython-313-arm-linux-gnueabi.so
# trying /root/msvcrt.abi3.so
# trying /root/msvcrt.so
# trying /root/msvcrt.py
# trying /root/msvcrt.pyc

The import checks are structured like so in _pyrepl/readline.py

try:
    from .unix_console import UnixConsole as Console, _error
except ImportError:
    from .windows_console import WindowsConsole as Console, _error

Maybe this should be:

if sys.platform == "win32":
    from .windows_console import WindowsConsole as Console, _error
else:
    from .unix_console import UnixConsole as Console, _error

It's not clear to me if cygwin is expected to work with the windows console or the unix console.

The register_readline handler already has similar branching logic:

                if os.name == "nt":
                    import _pyrepl.windows_console
                    console_errors = (_pyrepl.windows_console._error,)
                else:
                    import _pyrepl.unix_console
                    console_errors = _pyrepl.unix_console._error

The restructured import should allow the correct import error to bubble up and set CAN_USE_PYREPL=False.

# python3 -c "import _pyrepl.curses"
  File "<string>", line 1, in <module>
    import _pyrepl.curses
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/usr/lib/python3.13/_pyrepl/curses.py", line 28, in <module>
    from . import _minimal_curses as _curses  # type: ignore[no-redef]
  File "<frozen importlib._bootstrap>", line 1415, in _handle_fromlist
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/usr/lib/python3.13/_pyrepl/_minimal_curses.py", line 32, in <module>
    _clibpath = _find_clib()
  File "/usr/lib/python3.13/_pyrepl/_minimal_curses.py", line 28, in _find_clib
    traceback.print_stack()
Traceback (most recent call last):
  File "/usr/lib/python3.13/_pyrepl/curses.py", line 23, in <module>
    import _curses
ModuleNotFoundError: No module named '_curses'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/_pyrepl/curses.py", line 26, in <module>
    import curses as _curses  # type: ignore[no-redef]
    ^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'curses'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import _pyrepl.curses
  File "/usr/lib/python3.13/_pyrepl/curses.py", line 28, in <module>
    from . import _minimal_curses as _curses  # type: ignore[no-redef]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: cannot import name '_minimal_curses' from '_pyrepl' (/usr/lib/python3.13/_pyrepl/__init__.py)

The ModuleNotFoundError("curses library not found", name="_pyrepl._minimal_curses") is getting morphed into that ImportError above by from . import X.

def _find_clib() -> str:
    trylibs = ["ncursesw", "ncurses", "curses"]

    for lib in trylibs:
        path = ctypes.util.find_library(lib)
        if path:
            return path
    raise ModuleNotFoundError("curses library not found", name="_pyrepl._minimal_curses")

Dropping the name property seems to resolve that but maybe there's a better solution.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Activity

  1. vfazio
    added
    type-bugAn unexpected behavior, bug, or error
    on Feb 12, 2025
  2. tomasr8
    added
    topic-replRelated to the interactive shell
    on Feb 12, 2025

  3. tomasr8 commented on Mar 15, 2025

    @tomasr8
    tomasr8
    on Mar 15, 2025
    Member

    I just ran into this myself on a freshly installed Ubuntu 24.04 😆 I think a nicer error would definitely be useful. Would you like to send a PR @vfazio ?

  4. picnixz
    added
    stdlibStandard Library Python modules in the Lib/ directory
    on Mar 15, 2025

  5. vfazio commented on Mar 15, 2025

    @vfazio
    vfazio
    on Mar 15, 2025
    ContributorAuthor

    I just ran into this myself on a freshly installed Ubuntu 24.04 😆 I think a nicer error would definitely be useful. Would you like to send a PR @vfazio ?

    Yea ill try to whip something up tonight or tomorrow.

    One thing I was a little confused about is that it looks like we don't have a consistent way to check for "this is Windows". Both sys.platform and os.name are used throughout the project.

    I also never identified the machinery that was dropping the error string from importerror due to the name argument being specified.

    Shouldn't take long to do but who will review the changes? Or how do I get a reviewer assigned to the PR? One of my other PRs is kinda stuck in limbo too despite good conversation in the issue itself, so would like to avoid that here.

  6. tomasr8 commented on Mar 15, 2025

    @tomasr8
    tomasr8
    on Mar 15, 2025
    Member

    One thing I was a little confused about is that it looks like we don't have a consistent way to check for "this is Windows". Both sys.platform and os.name are used throughout the project.

    Not sure, it might require some testing. You could check which of windows_console or unix_console can be imported on cygwin.

    Or how do I get a reviewer assigned to the PR? One of my other PRs is kinda stuck in limbo too despite good conversation in the issue itself, so would like to avoid that here.

    pyrepl has a couple codeowners who get assigned automatically. This should be a relatively simple change, so we can also ping some other active core devs if needed.

  7. picnixz commented on Mar 15, 2025

    @picnixz
    picnixz
    on Mar 15, 2025
    Member

    I missed the fact that #131189 is a duplicate of this one. While we usually keep old ones opened and close new ones, let's do the reverse here as there's an opened PR out there.

  8. picnixz
    closed this as a duplicateof Compilation on Linux without both libffi-dev and libncurses-dev results in an unexpected warning when using the new REPL #131189 on Mar 15, 2025
  9. serhiy-storchaka
    added this to Curses issueson Jul 19, 2026
  10. serhiy-storchaka
    moved this to Done in Curses issueson Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions