Custom str subclass as keyword argument leads to System Error

When arguments are given by keyword using a dictionary, CPython requires that the key be of type str, or a sub-class. If a custom str subclass defines a badly-behaved __eq__, and is used that way, a SystemError (null argument) can result, which is not expected in the interpreter.

This report stems from a discussion on discuss.python.org about the types of keyword that ought to be allowed -- it's not code I actually want to run.

Reproducer

# kwargsbomb.py Use of a str subclass as a keyword

class S(str):
    def __eq__(self, other):
        print(other)
        return True
    def __hash__(self):
        return 42

u = "my.domain.name"

a = u.split(**{'sep':'.'})     # ok
print(a)

b = u.split(**{S('sep'):'.'})  # ok
print(b)

c = u.split(**{S('xxx'):'.'})  # fails
print(c)

Output

['my', 'domain', 'name']
['my', 'domain', 'name']
sep
Traceback (most recent call last):
  File " ... \kwargsbomb.py", line 18, in <module>
    c = u.split(**{S('xxx'):'.'})  # fails
        ^^^^^^^^^^^^^^^^^^^^^^^^^
SystemError: null argument to internal routine

Environment

CPython versions: 3.10.2 3.11.0b4 (in Idle and at the prompt)
Operating system and architecture: Windows 10 64-bit

I'm not sure this should be supported. The discuss.python.org discussion was focused on strings that aren't valid identifiers, not on subclasses of str.

Restricting kwargs to only str may allow some optimizations and saves us from possible subtle bugs (what if a str subclass overrides __hash__). In addition, there's no known use case for using str subclasses.

I suggest we:

Document that kwargs must be exactly of type str
Fix the code so that attempting to use a str subclass raises TypeError instead of SystemError

The user should not be able to provoke a SystemError from pure Python code. This exception is reserved for misuse of the C API. There is a bug somewhere in _PyArg_UnpackKeywords(). In other examples it can lead to crash, memory corruption, or just silent produce incorrect result.

@serhiy-storchaka

The discuss.python.org discussion was focused on strings that aren't valid identifiers, not on subclasses of str.

It starts with the question why integer keys are not allowed. When I looked where this is enforced, I was interested to find that it accepted sub-classes of str. So I tried it, idly, and idle restarted (in 3.8).

Restricting kwargs to only str may allow some optimizations and saves us from possible subtle bugs (what if a str subclass overrides __hash__).

I agree, and favour that, but I cannot say for certain no-one is relying on it. Also @serhiy-storchaka is right that something is wrong elsewhere, and may go wrong in other circumstances, which this daft trick just serves to discover.

pythongh-94938: Fix errror detection of unexpected keyword arguments

@jeff5

Thank you for your report @jeff5. It was an interesting bug.

gh-94938: Fix errror detection of unexpected keyword arguments (GH-94999

[3.11] pythongh-94938: Fix errror detection of unexpected keyword arg…

[3.10] pythongh-94938: Fix errror detection of unexpected keyword arg…

[3.11] gh-94938: Fix errror detection of unexpected keyword arguments (…

[3.10] gh-94938: Fix errror detection of unexpected keyword arguments (…

pythongh-94938: Fix test

gh-94938: Fix test (GH-95396)

pythongh-94938: Fix test (pythonGH-95396)

gh-94938: Fix test (GH-95396)

Navigation Menu

Repository navigation

Custom str subclass as keyword argument leads to System Error #94938

Reproducer

Output

Environment

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Custom str subclass as keyword argument leads to System Error #94938

Description

Reproducer

Output

Environment

Activity

JelleZijlstra commented on Jul 17, 2022

serhiy-storchaka commented on Jul 18, 2022

jeff5 commented on Jul 19, 2022

serhiy-storchaka commented on Jul 19, 2022

merwok commented on Aug 29, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions