Repository navigation
Typing: improve error messages for invalid ParamSpec substitutions #102725
Description
Activity
Update: turns out the later substitution does not raise an error anymore on
main:import typing as t, collections.abc as c P, T = t.ParamSpec("P"), t.TypeVar("T") class MyCallable(t.Generic[P, T]): ... print(MyCallable[P, T][[P, str], bool][int]) # __main__.MyCallable[[[int], str], bool]
Not good!
I think that this error is helpful and should be restored with a better message.
We can now even substitute
Twith[P, int]:import typing as t, collections.abc as c P, T, T1 = t.ParamSpec("P"), t.TypeVar("T"), t.TypeVar("T1") class MyCallable(t.Generic[T1, T]): ... print(MyCallable[T, T1][[P, str], bool]) # __main__.MyCallable[[~P, <class 'str'>], bool]
Not good at all!
Should we fix it? It will require quite a bit of work to get all the things right.
I can do it if we need it. Or we can keep this up to a user to control.If it can be fixed without excessively complicating the code of
typing.py, and if we can be 100% sure that we're not introducing false-positive errors, then I'll be happy to review a PR. But our general position is that it's better for the runtime to err on the side of leniency, and leave it to type checkers to point out invalid uses oftypingconstructs. We've had lots of situations in the past where strict checks at runtime have madetyping_extensionsbackports complicated, or interfered with people using type annotations in unusual/experimental ways. Therefore, I view these "false negatives" at runtime as being quite low-priority concerns.Let's start with the second example.
The change itself is quite easy:if (isinstance(old_arg, ParamSpec) and isinstance(new_arg, tuple) and any(isinstance(na, ParamSpec) for na in new_arg)): raise TypeError( 'Cannot replace a ParamSpec with ' 'a tuple containing other ParamSpec, ' 'use Concatenate[] instead' ) elif self.__origin__ == collections.abc.Callable and isinstance(new_arg, tuple):
But, it has several flaws.
collections.abc.Callabledoes not do the same- It solves a very specific problem most users don't have
- Type-checker can catch this just fine: https://mypy-play.net/?mypy=latest&python=3.11&gist=627d3f8590da64c7c9e077067fd1b8e8
main.py:6: error: Invalid location for ParamSpec "P" [valid-type] main.py:6: note: You can use ParamSpec as the first argument to Callable, e.g., 'Callable[P, int]'So, I think we should leave this as-is.
Now, let's get back to your first example.
>>> import typing as t, collections.abc as c >>> P, T = t.ParamSpec("P"), t.TypeVar("T") >>> c.Callable[P, T][[P, str], bool][int] Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<frozen _collections_abc>", line 501, in __getitem__ File "<frozen _collections_abc>", line 456, in __new__ TypeError: Callable must be used as Callable[[arg, ...], result].
It is also problematic, because right now
typing.Callabledoes not produce this error:>>> import typing as t, collections.abc as c >>> P, T = t.ParamSpec("P"), t.TypeVar("T") >>> t.Callable[P, T][[P, str], bool][int] typing.Callable[[int, str], bool]
Synchronizing them will be a huge pain.
Because the exception happens on[int]substitution. But, we have to make an invalid substitution herec.Callable[P, T][[P, str], bool]to be able to reach this state.So, I think we should just close this issue.
There's nothing that can be easily done to solve a hypothetical issue.Reacted by Alex Waygood
Feature or enhancement
PEP-612 is a complex typing PEP that introduced some complex rules, both for static type checkers and for CPython at runtime. It can be hard to remember the intricacies of all these rules -- both for users and for typing maintainers.
One thing that might help here is if we had better error messages in situations where the runtime raises exceptions. For example, in the following snippet, the runtime is arguably correct in raising
TypeErrors -- by the rules laid out in PEP-612, both of these are invalid substitutions for two reasons:ParamSpecin a parameters list usingtyping(_extensions).Concatenate.ParamSpecin a parameters list is disallowed.However, the error messages don't mention either of these things, and are in fact pretty baffling:
Previous discussion
This issue is a followup to some points raised in:
Cc. @sobolevn