Skip to content

IndentationError in new repl #133541

Closed
Closed
IndentationError in new repl#133541
Assignees
ambv
Labels
3.14bugs and security fixesstdlibStandard Library Python modules in the Lib/ directorytopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

Description

@ronaldoussoren
ronaldoussoren
opened on May 6, 2025

Bug report

Bug description:

Python version: Python 3.14.0a7+ (heads/main-dirty:8d0e07eb899.

To reproduce type the following class definition in the REPL:

class C:
    def
    m

Then set the cursor to the space before m and press backspace to correct the incorrect newline.

Behaviour: the REPL exits with an IndentationError.

Typing backspace works correctly when the cursor is on m.

The exit/crash does not happen in 3.14a7.

CPython versions tested on:

3.14, CPython main branch

Operating systems tested on:

macOS

Linked PRs

Activity

  1. ronaldoussoren
    added
    type-bugAn unexpected behavior, bug, or error
    on May 6, 2025

  2. terryjreedy commented on May 7, 2025

    @terryjreedy
    terryjreedy
    on May 7, 2025
    Member

    Win 10: with a space between the cursor and 'm', so 3 spaces to left of cursor
    IndentationError: unindent does not match any outer indentation level

  3. ronaldoussoren commented on May 7, 2025

    @ronaldoussoren
    ronaldoussoren
    on May 7, 2025
    ContributorAuthor

    Full traceback:

    Python 3.14.0a7+ (heads/main-dirty:8d0e07eb899, May  6 2025, 18:45:02) [Clang 17.0.0 (clang-1700.0.13.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class C:
...     def
...    Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
                     "__main__", mod_spec)
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/__main__.py", line 10, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/simple_interact.py", line 138, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/readline.py", line 393, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 751, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 734, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 664, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 641, in refresh
    self.screen = self.calc_screen()
                  ~~~~~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/completing_reader.py", line 261, in calc_screen
    screen = super().calc_screen()
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 315, in calc_screen
    colors = list(gen_colors(self.get_unicode()))
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/utils.py", line 97, in gen_colors
    for color in gen_colors_from_token_stream(gen, line_lengths):
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/utils.py", line 155, in gen_colors_from_token_stream
    for prev_token, token, next_token in token_window:
                                         ^^^^^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/utils.py", line 348, in prev_next_window
    for x in iterator:
             ^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/tokenize.py", line 586, in _generate_tokens_from_c_tokenizer
    raise e from None
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/tokenize.py", line 582, in _generate_tokens_from_c_tokenizer
    for info in it:
                ^^
  File "<string>", line 3
    method(self): pass
                      ^
IndentationError: unindent does not match any outer indentation level
  4. ronaldoussoren
    added
    topic-replRelated to the interactive shell
    on May 7, 2025
  5. picnixz
    added
    stdlibStandard Library Python modules in the Lib/ directory
    3.14bugs and security fixes
    on May 7, 2025

  6. StanFromIreland commented on May 7, 2025

    @StanFromIreland
    StanFromIreland
    on May 7, 2025
    Member

    I can reproduce but with a varied traceback.

    Python 3.14.0a7+ (heads/main:3dfed230928, May  6 2025, 17:44:25) [GCC 15.1.1 20250425 (Red Hat 15.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class C:
...     def
...    Traceback (most recent call last):
  File "/usr/local/lib/python3.14/_pyrepl/readline.py", line 393, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 748, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 731, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 661, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 638, in refresh
    self.screen = self.calc_screen()
                  ~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/_pyrepl/completing_reader.py", line 261, in calc_screen
    screen = super().calc_screen()
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 315, in calc_screen
    colors = list(gen_colors(self.get_unicode()))
  File "/usr/local/lib/python3.14/_pyrepl/utils.py", line 102, in gen_colors
    for color in gen_colors_from_token_stream(gen, line_lengths):
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/_pyrepl/utils.py", line 160, in gen_colors_from_token_stream
    for prev_token, token, next_token in token_window:
                                         ^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/_pyrepl/utils.py", line 357, in prev_next_window
    for x in iterator:
             ^^^^^^^^
  File "/usr/local/lib/python3.14/tokenize.py", line 586, in _generate_tokens_from_c_tokenizer
    raise e from None
  File "/usr/local/lib/python3.14/tokenize.py", line 582, in _generate_tokens_from_c_tokenizer
    for info in it:
                ^^
  File "<string>", line 3
    m
     ^
IndentationError: unindent does not match any outer indentation level

  7. danielhollas commented on May 7, 2025

    @danielhollas
    danielhollas
    on May 7, 2025
    Contributor

    cc @ambv based on the stacktrace it looks like it's coming from the new syntax highlighting code.

  8. ambv
    self-assigned this
    on May 7, 2025

  9. ambv commented on May 7, 2025

    @ambv
    ambv
    on May 7, 2025
    Contributor

    Oof. OK, yes, indeed. Sorry about that omission.

  10. ambv
    added a commit that references this issue on May 7, 2025

    pythongh-133541: Handle SyntaxError raised by the tokenizer on user i…

    431b928
  11. pablogsal
    added a commit that references this issue on May 8, 2025

    gh-133541: Handle SyntaxError raised by the tokenizer on user input (#…

    b2fabce
  12. miss-islington
    added a commit that references this issue on May 8, 2025

    pythongh-133541: Handle SyntaxError raised by the tokenizer on user i…

    1971073
  13. pablogsal
    added a commit that references this issue on May 8, 2025

    [3.14] gh-133541: Handle SyntaxError raised by the tokenizer on user …

    0c340d1

  14. ferdnyc commented on May 13, 2025

    @ferdnyc
    ferdnyc
    on May 13, 2025
    Contributor

    Is this issue fixed, now, with the merge of #133606? Sounds like it.

  15. StanFromIreland commented on May 13, 2025

    @StanFromIreland
    StanFromIreland
    on May 13, 2025
    Member

    I can confirm it has been fixed, the linked repro no longer works. This can be closed.

  16. terryjreedy
    closed this as completedon May 13, 2025

  17. terryjreedy commented on Jun 9, 2025

    @terryjreedy
    terryjreedy
    on Jun 9, 2025
    Member

    #135185 closed as a duplicate of this.

  18. terryjreedy
    marked Python 3.14 REPL sometimes breaks when editing code blocks #135185 as a duplicate of this issue on Jun 9, 2025
  19. Pranjal095
    added a commit that references this issue on Jul 12, 2025

    pythongh-133541: Handle SyntaxError raised by the tokenizer on user i…

    92e24ab
  20. taegyunkim
    added a commit that references this issue on Aug 4, 2025

    pythongh-133541: Handle SyntaxError raised by the tokenizer on user i…

    f7d1d50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.14bugs and security fixesstdlibStandard Library Python modules in the Lib/ directorytopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions