typing._type_check no longer recognizes several kinds of non-types

Bug report

The internal function typing._type_check() is used extensively to check whether a particular value represents a type. Its docstring begins with:

def _type_check(arg, msg, is_argument=True, module=None, *, allow_special_forms=False):
    """Check that the argument is a type, and return it (internal helper).

In Python 3.11b1 this function no longer errors on various kinds of non-types:

>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=11, micro=0, releaselevel='beta', serial=1)
>>> import typing
>>> typing._type_check(5, "Expected a type.")
# no error!
>>> typing.ClassVar[5]
typing.ClassVar[5]  # no error!

By contrast on Python 3.10:

>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=10, micro=1, releaselevel='final', serial=0)
>>> import typing
>>> typing._type_check(5, "Expected a type.")
TypeError: Expected a type. Got 5.
>>> typing.ClassVar[5]
TypeError: typing.ClassVar accepts only single type. Got 5.

Your environment

Regression introduced somewhere between Python 3.10 and Python 3.11b1.

@JelleZijlstra

CC @JelleZijlstra

I think commit 870b22b changed this behavior. It's intentional. Please see the explanation at #31151 (review).

@davidfstr

I think commit 870b22b changed this behavior. It's intentional.

Understood. I'm currently using typing._type_check() inside the trycast library - which is a runtime type checker - to provide better error messages to users who mistakenly pass a regular Python value where a typeform is expected. For example the following common case:

trycast(int, 5)  # OK
trycast(5, int)  # ERROR: TypeError: trycast() requires a type as its first argument. Got 5.

It looks like typing._type_check() was intentionally loosened to not require the maybe-typeform it was checking to be callable so that:

It was understood that loosening this check would cause _type_check() to potentially deviate from how static type checkers recognize a "type", but that giving additional flexibility to the runtime typing implementation was considered to be a good tradeoff.

My particular usage of typing._type_check() in trycast desires to only allow garden-variety/universal typeforms (that are generally useable anywhere) but not "only-allowed-in-special-places" typeforms like ParamSpecArgs or InitVar. And it also doesn't want to allow "definitely-not-typeforms" like values of {int, float, bool} (ex: 5).

I would argue that questionable constructions like ClassVar[5] also would benefit from disallowing "definitely-not-typeforms" as well, as they did in Python ≤ 3.10.

Suggested remediations at the typing.py layer:

Revert typing._type_check() to only permit "universal" typeforms (than can be used anywhere) by default, and have things like Annotated[] explicitly allow non-universal typeforms via something like typing._type_check(maybe_typeform, allow_nonuniversal=False).
- Pro: The majority of typing._type_check() callers revert to disallowing non-typeforms and non-universal typeforms, with certain special callers like Annotated[] being able to opt-in to them.
- Detail: Ways that a "universal" typeform could be recognized:
  - Proposal 1: callable(maybe_typeform) # former status-quo
  - Proposal 2: isinstance(maybe_typeform, (type, typing._SpecialForm))
  - Proposal 3: isinstance(maybe_typeform, typing._UniversalSpecialForm), where typing._UniversalSpecialForm is a new ABC, which at least {type, typing._SpecialForm} implement
Add a deny-list to typing._type_check() to guard against "definitely-not-typeforms" like values of {int, float, bool}
- Pro: Provides better best-effort recognition (and denial) of non-typeforms than the current implementation.
- Con: A deny-list will necessarily be incomplete.
- Con: A deny-list, since it will add potentially a lot of extra checks, will be slow to execute at runtime.

Suggested remediations at the trycast layer (for @davidfstr to consider if no remediations at the typing.py layer are accepted):

Alter trycast to use a wrapper around typing._type_check() that reverts it to the Python ≤3.10 behavior of disallowing non-callables (which is currently a very reasonable way to filter out non-universal typeforms).

@GBeauregard

CC @GBeauregard as the originator of bpo-46644
CC @AlexWaygood as a +1er to @JelleZijlstra 's comment on that PR to accept it

This was an intentional change to reduce the number of bugs where typing.py would reject current or future exotic but valid type annotations.

Also, _type_check is an internal implementation detail, subject to change at any time. It's also not very good at recognizing valid type annotations: things like ClassVar[len] were always accepted. You should implement your own, more precise, check for valid types in your library.

You should implement your own, more precise, check for valid types in your library.

Will do. Thanks for taking a look!

Navigation Menu

typing._type_check no longer recognizes several kinds of non-types #92601

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

typing._type_check no longer recognizes several kinds of non-types #92601

Description

Activity

davidfstr commented on May 10, 2022

Fidget-Spinner commented on May 10, 2022

davidfstr commented on May 10, 2022

davidfstr commented on May 10, 2022

JelleZijlstra commented on May 10, 2022

davidfstr commented on May 11, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions