mypy does not handle __class_getitem__

PEP 560 defines __class_getitem__ as a way to enable indexing a class (to define custom generic types for instance), but mypy does not seem to recognize this.

To Reproduce

$ cat test.py
class A:
    def __class_getitem__(cls, item):
        return cls

x = A[int]

$ mypy test.py

Expected Behavior

Success: no issues found in 1 source file

Actual Behavior

test.py:6: error: "A" expects no type arguments, but 1 given
Found 2 errors in 1 file (checked 1 source file)

Environment

mypy 0.910
Python 3.9.7
Darwin arm64

Oh wow! I always thought it was possible!

PR with the fix is on its way. I hope, that I won't break things.

Ok, I misunderstood your problem a bit at first. Please, let me explain what is going on.

class A:
    def __class_getitem__(cls, item):
        return cls

x = A[int]

Here A[int] is treated as "type application" by mypy. This is the core of how we work with Generic types. Like List[int] or YourGeneric[T]. Moreover, __class_getitem__ was added especially for this use-case. It is designed to be representing generic type args.

Mypy here complains about missing Generic[T] base class in your definition. Because we associate "type application" not with __class_getitem__, but with Generic (which provides this magic method to an object).

What is your use-case for bare __class_getitem__?

I was trying to implement a generic-like interface indexed with strings. Something like:

GenCls[“spec”]

where new types are constructed at run time based on the string. If I’m not wrong, this isn’t possible with Generic as a base class, since the index isn’t a type.

Is __class_getitem__ not meant to be used directly?

I should add: type checking also breaks if using a metaclass with __getitem__, instead of __class_getitem__. I think it makes sense to treat obj[key] as just regular indexing if obj is not a Generic subclass.

This might be a good idea! I will try to send a prototype PR today. We can start a further discussion from there.

It might be a good idea for this to be fixed in mypy, as the language doesn't prevent you from using __class_getitem__ for non-type-checking purposes.

It may be worth noting, however, that the documentation for __class_getitem__ (newly rewritten by me 😉) does include the following warning regarding __class_getitem__:

Custom implementations of class_getitem() on classes defined outside of the standard library may not be understood by third-party type-checkers such as mypy. Using class_getitem() on any class for purposes other than type hinting is discouraged.

@AlexWaygood

@AlexWaygood see my #11558 PR, it has lots of problems. I am not sure that this is actually worth the effort.

@jayanthkoushikim in what way does "type checking also break if using a metaclass with __getitem__, instead of __class_getitem__" 🙂? This seems to work fine:

from typing import TypeVar, Any
T = TypeVar('T')

class Meta(type):
    def __getitem__(cls: type[T], arg: Any) -> type[T]:
        return cls
        
class Foo(metaclass=Meta): ...
reveal_type(Foo["Does this work?"]) # Revealed type is "Type[__main__.Foo*]"

@AlexWaygood

@AlexWaygood Hm, interesting that reveal type works. But assigning Foo["..."] to anything, or using it as a type annotation does not work.

$ cat test.py
from typing import TypeVar, Any
T = TypeVar('T')

class Meta(type):
    def __getitem__(cls: type[T], arg: Any) -> type[T]:
        return cls
    
class Foo(metaclass=Meta): ...
reveal_type(Foo["Does this work?"])
x = Foo["Does this work?"]
y: Foo["Does this work?"]

$ mypy test.py
test.py:9: note: Revealed type is "Type[test.Foo*]"
test.py:10: error: "Foo" expects no type arguments, but 1 given
test.py:10: error: Invalid type comment or annotation
test.py:11: error: "Foo" expects no type arguments, but 1 given
test.py:11: error: Invalid type comment or annotation
Found 5 errors in 1 file (checked 1 source file)

@jayanthkoushik

@jayanthkoushik fair enough!

IMO, at least x = Foo[…] should be supported since it does not use any undocumented or unrecommended behavior.

IMO, at least x = Foo[…] should be supported since it does not use any undocumented or unrecommended behavior.

Yes, I think I agree that this would be very nice to have in the case of metaclass __getitem__. For __class_getitem__, I'm much more ambivalent, since, at the end of the day, it was only ever really intended to be used for classes inside the stdlib.

It looks like master (at least) is already capable of understanding this code:

class Meta(type):
    def __getitem__(cls, arg: str) -> 'Meta':
        return cls

class My(metaclass=Meta):
    ...

reveal_type(My[1])
# out/ex.py:8: note: Revealed type is "ex.Meta"
# out/ex.py:8: error: Invalid index type "int" for "Type[My]"; expected type "str"

Yeah, see #2827 and #1771 for the fix and implementation of __getitem__ support in metaclasses.

To add a case to this, it would be cool if we could leverage a custom __class_getitem__ while also retaining the typing benefits of Generic. Something like this:

from typing import ClassVar, Generic, TypeVar

T = TypeVar("T")

class Foo(Generic[T]):
    cls_attr: ClassVar[int]

    def __class_getitem__(cls, item: tuple[int, T]):
        cls.cls_attr = item[0]
        return super().__class_getitem__(item[1])

    def __init__(self, arg: T):
        self.arg = arg

foo = Foo[1, bool](arg=True)
assert foo.cls_attr == 1

I have created a library to generate partial models for pydantic (https://github.com/team23/pydantic-partial). The basic idea would be to have something like Partial[Foo] to create a copy of Foo where all fields allow None and have a default value of None. This tries to follow what Typescript does, see https://www.typescriptlang.org/docs/handbook/utility-types.html#partialtype for reference.

Currently I am using a function to generate the partial model instead of Partial.__class_getitem__ as mypy does not understand what is going on here. Thus I wanted to add this use case here, as it even strictly follows the original intent of the __class_getitem__ method to be used for typing. So it would be really cool to support this and allow typing helpers to work with __class_getitem__, too.

As an additional note: I'm totally aware that I am generating dynamic types here which will pose additional headaches. I'm currently in the progress of investigating how to "tell" mypy that my current approach creates a valid type (error like: "Unsupported dynamic base class"). But this is a totally different story and may be fixable with creating a mypy plugin (I'm possibly going for this, soon). I just stumbled over this issue and thought to add this idea here - as I really would like Partial[...] to work like Partial<...> in Typescript. ;-)

@AlexWaygood

I'm sorry that I'm asking here, but the question I want to ask is strictly connected to the subject of this issue.

Correct me if I'm wrong, but from what I understood, __class_getitem__ serves for type hinting right? That is what I understood from description which has been recently updated by @AlexWaygood and also PEP 560.

Now question is does it concern generic core abstract classes as well? Because I have this feature request #14537 in which I have given an example for which mypy is returning errors:

multi.py:26: error: Incompatible types in assignment (expression has type "List[str]", variable has type "MyIterable[str]")  [assignment]
multi.py:27: error: Incompatible types in assignment (expression has type "List[List[str]]", variable has type "MyIterable[MyIterable[str]]")  [assignment]
multi.py:40: error: Argument 3 to "get_enum_str" has incompatible type "List[str]"; expected "MyIterable[str]"  [arg-type]
multi.py:41: error: Argument 2 to "get_enum_str_list" has incompatible type "List[str]"; expected "MyIterable[str]"  [arg-type]
multi.py:41: error: Argument 3 to "get_enum_str_list" has incompatible type "List[List[str]]"; expected "MyIterable[MyIterable[str]]"  [arg-type]
multi.py:42: error: Argument 2 to "get_enum_str_list" has incompatible type "str"; expected "MyIterable[str]"  [arg-type]
multi.py:42: error: Argument 3 to "get_enum_str_list" has incompatible type "List[List[str]]"; expected "MyIterable[MyIterable[str]]"  [arg-type]

Navigation Menu

Repository navigation

mypy does not handle __class_getitem__ #11501

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

mypy does not handle __class_getitem__ #11501

Description

Activity

sobolevn commented on Nov 9, 2021

sobolevn commented on Nov 9, 2021

jayanthkoushik commented on Nov 9, 2021

jayanthkoushik commented on Nov 9, 2021

sobolevn commented on Nov 9, 2021

AlexWaygood commented on Nov 17, 2021

sobolevn commented on Nov 17, 2021

AlexWaygood commented on Nov 17, 2021

jayanthkoushik commented on Nov 17, 2021

AlexWaygood commented on Nov 17, 2021

jayanthkoushik commented on Nov 17, 2021

AlexWaygood commented on Nov 17, 2021

sobolevn commented on Nov 18, 2021

dmadisetti commented on Jan 11, 2022

jamesbraza commented on Sep 1, 2022

ddanier commented on Sep 29, 2022

Dvergatal commented on Jan 27, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions