New REPL on Windows exits when accented character is pasted/typed

Crash report

What happened?

When pasting an accented character like ñ or é, or typing one such as ´p and pressing Enter, then up arrow, the new REPL on Windows will exit with the following error:

Python 3.14.0a6+ (heads/fix_non_string_candidates_exit_repl-dirty:8008a588342, Mar 29 2025, 08:15:) [MSC v.1943 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\ddini\PycharmProjects\cpython\Lib\_pyrepl\__main__.py", line 6, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\ddini\PycharmProjects\cpython\Lib\_pyrepl\main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "C:\Users\ddini\PycharmProjects\cpython\Lib\_pyrepl\simple_interact.py", line 137, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "C:\Users\ddini\PycharmProjects\cpython\Lib\_pyrepl\readline.py", line 389, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "C:\Users\ddini\PycharmProjects\cpython\Lib\_pyrepl\reader.py", line 750, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "C:\Users\ddini\PycharmProjects\cpython\Lib\_pyrepl\reader.py", line 706, in handle1
    event = self.console.get_event(block=False)
  File "C:\Users\ddini\PycharmProjects\cpython\Lib\_pyrepl\windows_console.py", line 471, in get_event
    self.event_queue.push(rec.Event.KeyEvent.uChar.UnicodeChar)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ddini\PycharmProjects\cpython\Lib\_pyrepl\base_eventqueue.py", line 88, in push
    assert len(self.buf) == 1
           ^^^^^^^^^^^^^^^^^^
AssertionError

In these cases, len(self.buf) is 3 instead of 1.

I'm running Python in Windows Terminal, chcp gives 850, changing it to 65001 (UTF-8) makes no difference.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Output from running 'python -VV' on the command line:

Python 3.14.0a6+ (heads/main-dirty:7c3692fe275, Mar 24 2025, 19:20:37) [MSC v.1942 64 bit (AMD64)]

Linked PRs

Related? #130273

This is a bug and not a crash.

@DinoV

This is a bug and not a crash.

Correct, sorry about that.

cc @DinoV, @y5c4l3, @encukou

This is os-windows specific. IIRC there was recently an issue about this, it was closed.

Yes looking at #130804 it seems it was not fully resolved, and it is presumably a side effect of the merged pr, do we want to reopen that?

@StanFromIreland

@StanFromIreland
It is in another execution path, so I think this should be a new issue (despite it looks similar).

@sergey-miryanov

@sergey-miryanov

cpython/Lib/_pyrepl/base_eventqueue.py

Lines 72 to 89 in fccf9ab

    
               def push(self, char: int | bytes | str) -> None: 
        
                   """ 
        
                   Processes a character by updating the buffer and handling special key mappings. 
        
                   """ 
        
                   ord_char = char if isinstance(char, int) else ord(char) 
        
                   if ord_char > 255: 
        
                       assert isinstance(char, str) 
        
                       char = bytes(char.encode(self.encoding, "replace")) 
        
                       self.buf.extend(char) 
        
                   else: 
        
                       char = bytes(bytearray((ord_char,))) 
        
                       self.buf.append(ord_char) 
        
                   if char in self.keymap: 
        
                       if self.keymap is self.compiled_keymap: 
        
                           # sanity check, buffer is empty when a special key comes 
        
                           assert len(self.buf) == 1 
        
                       k = self.keymap[char]

It is in the same path you modified, your changes are the last ones to modify the buffer before the check?

@StanFromIreland

@StanFromIreland Oh, I get what you mean. Yes, it is related, but I think it is related to keymap and compiled_keymap also, so it extends my previous patch a bit. I will test and fix. Can you assign it to me?

Also, I should mention that it is not backported to 3.13 due conflicts - #130805 (comment)

Merge branch 'main' into pythongh-131878-fix-unicode-input-in-new-pyr…

gh-131878: Handle top level exceptions in new pyrepl and prevent of c…

[3.13] pythongh-131878: Handle top level exceptions in new pyrepl and…

[3.13] gh-131878: Handle top level exceptions in new pyrepl and preve…

gh-131878: Fix input of unicode characters with two or more code poin…

Fixed, thank you all!

[3.13] pythongh-131878: Fix input of unicode characters with two or m…

[3.13] gh-131878: Fix input of unicode characters with two or more co…

pythongh-131878: Handle top level exceptions in new pyrepl and preven…

pythongh-131878: Fix input of unicode characters with two or more cod…

Navigation Menu

Repository navigation

New REPL on Windows exits when accented character is pasted/typed #131878

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

4 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	def push(self, char: int \| bytes \| str) -> None:
	"""
	Processes a character by updating the buffer and handling special key mappings.
	"""
	ord_char = char if isinstance(char, int) else ord(char)
	if ord_char > 255:
	assert isinstance(char, str)
	char = bytes(char.encode(self.encoding, "replace"))
	self.buf.extend(char)
	else:
	char = bytes(bytearray((ord_char,)))
	self.buf.append(ord_char)

	if char in self.keymap:
	if self.keymap is self.compiled_keymap:
	# sanity check, buffer is empty when a special key comes
	assert len(self.buf) == 1
	k = self.keymap[char]

New REPL on Windows exits when accented character is pasted/typed #131878

Description

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Activity

StanFromIreland commented on Mar 29, 2025

devdanzin commented on Mar 29, 2025

StanFromIreland commented on Mar 29, 2025

StanFromIreland commented on Mar 29, 2025

sergey-miryanov commented on Mar 29, 2025

StanFromIreland commented on Mar 29, 2025

sergey-miryanov commented on Mar 29, 2025

sergey-miryanov commented on Mar 29, 2025

4 remaining items

ambv commented on May 5, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions