TypeForm[T]: Spelling for regular types (int, str) & special forms (Union[int, str], Literal['foo'], etc)

@ltworf

(An earlier version of this post used TypeAnnotation rather than TypeForm as the initially proposed spelling for the concept described here)

Feature

A new special form TypeForm[T] which is conceptually similar to Type[T] but is inhabited by not only regular types like int and str, but also by anything "typelike" that can be used in the position of a type annotation at runtime, including special forms like Union[int, str], Literal['foo'], List[int], MyTypedDict, etc.

Pitch

Being able to represent something like TypeForm[T] enables writing type signatures for new kinds of functions that can operate on arbitrary type annotation objects at runtime. For example:

# Returns `value` if it conforms to the specified type annotation using typechecker subtyping rules.
def trycast(typelike: TypeForm[T], value: object) -> Optional[T]: ...

# Returns whether the specified value can be assigned to a variable with the specified type annotation using typechecker subtyping rules.
def isassignable(value: object, typelike: TypeForm[T]) -> bool: ...

Several people have indicated interest in a way to spell this concept:

For a more in-depth motivational example showing how I can use something like TypeForm[T] to greatly simplify parsing JSON objects received by Python web applications, see my recent thread on typing-sig:

Spelling the type of a runtime type annotation object (typing-sig thread)

If there is interest from the core mypy developers, I'm willing to do the related specification and implementation work in mypy.

Why not do something like:

from __future__ import annotations
from typing import *

T = TypeVar("T")

class TypeAnnotation(Generic[T]):
    @classmethod
    def trycast(cls, value: object) -> T:
        ...

reveal_type(TypeAnnotation[Optional[int]].trycast(object()))

@hauntsaninja

Why can't we make Type[T] also work for other special forms?

@hauntsaninja's workaround is useful, but it would be better to have a feature in the core type system for this.

@hauntsaninja

@hauntsaninja , using your workaround I am unable to access the passed parameter at runtime from inside the wrapper class.

The following program:

from __future__ import annotations
from typing import *

T = TypeVar('T')

class TypeAnnotation(Generic[T]):
    @classmethod
    def trycast(cls, value: object) -> Optional[T]:
        Ts = get_args(cls)
        print(f'get_args(cls): {Ts!r}')
        return None

ta = TypeAnnotation[Union[int, str]]
print(f'get_args(ta): {get_args(ta)!r}')
result = ta.trycast('a_str')

prints:

get_args(ta): (typing.Union[int, str],)
get_args(cls): ()

@JelleZijlstra

@JelleZijlstra commented:

Why can't we make Type[T] also work for other special forms?

@hauntsaninja's workaround is useful, but it would be better to have a feature in the core type system for this.

I agree that having Type[T] be widened to mean "anything typelike, including typing special forms" would be an alternate solution. A very attractive one IMHO.

However it appears that there was a deliberate attempt in mypy 0.780 to narrow Type[T] to only refer to objects that satisfy isinstance(x, type) at runtime. I don't understand the context for that decision. If however that decision was reversed and Type[T] made more general then there would be no need for the additional TypeAnnotation[T] syntax I'm describing in this issue.

I believe the issue is that Type is used for things that can be used as the second argument of isinstance(). And those things must be actual class objects (or tuples of such) -- they cannot be things like Any, Optional[int] or List[str].

So if this feature is going to happen I think it should be a separate thing -- and for the static type system it probably shouldn't have any behavior, since such objects are only going to be useful for introspection at runtime. (And even then, how are you going to do the introspection? they all have types that are private objects in the typing module.)

Well I wrote this module with various checks and add every new major py version to the tests to see that it keeps working: https://github.com/ltworf/typedload/blob/master/typedload/typechecks.py

Luckily since py3.6 it has not happened that the typing objects change between minor upgrades of python.

I believe the issue is that Type is used for things that can be used as the second argument of isinstance(). And those things must be actual class objects (or tuples of such) -- they cannot be things like Any, Optional[int] or List[str].

Makes sense.

for the static type system it probably shouldn't have any behavior, since such objects are only going to be useful for introspection at runtime.

Agreed.

how are you going to do the introspection? they all have types that are private objects in the typing module.

The typing module itself provides a few methods that can be used for introspection. typing.get_args and typing.get_origin come to mind.

I could see a couple of possible spellings for the new concept:

TypeAnnotation (as proposed earlier by the issue title)
TypeForm -- as in a "typing special form", which is consistent with the following existing documentation:
- "special forms" -- https://docs.python.org/3/library/typing.html#special-forms
- "special typing form" -- https://docs.python.org/3/library/typing.html#typing.get_origin
TypeHint -- as in the original PEP 484 "Type Hints", although I still lean toward "type annotation" as more of the proper full name

Personally I'm now leaning toward TypeForm (over TypeAnnotation) because it is consistent with prior documentation and is more succinct to type. It does sound a bit abstract but I expect only relatively advanced developers will be using this concept anyway.

(Let the bikeshedding begin. :)

I like TypeForm.

Okay I'll go with TypeForm then, for lack of other input.

Next steps I expect are for me to familiarize myself with the mypy codebase again since I'm a bit rusty. Hard to believe it's been as long as since 2016 I put in the first version of TypedDict. Rumor is it that the semantic analyzer has undergone some extensive changes since then.

Good. And yeah, a lot has changed. Once you have this working we should make a PEP out of it.

Once you have this working we should make a PEP out of it.

Yep, will do this time around. :)

Update: I redownloaded the high-level mypy codebase structure this afternoon to my brain. It appears there are now only 4 major passes of interest:

State.semantic_analysis_pass1()
semanal_main.semantic_analysis_for_scc() # pass 2
TypeChecker.check_first_pass()
TypeChecker.check_second_pass()

Next steps I expect are to trace everywhere that mypy is processing occurrences of Type[T] and T = TypeVar('T'), which I expect to be most-similar in implementation to the new TypeForm[T] support.

renaming TypeForm[] to TypeExpression[]

We could also consider TypeExpr for brevity; "expr" is a fairly common and well-understood abbreviation of "expression".

I wonder if defining a TypeExpression[] as an ABC would make it easier for a static type checker to recognize

I don't think that would work very well. The fact that Union[str, int] is an instance of _UnionGenericAlias at runtime is an implementation detail that type checkers should not be aware of. Even if we make TypeExpression an ABC, type checkers would still need special-casing to know that Union[str, int] is compatible with TypeExpression.

@davidfstr

@davidfstr I think this discussion should be happening on discourse as it affects more than just mypy. As for the concern Eric brought up, that was (part of) the reason for what I mentioned with TypeForm[T, *Args] earlier in this thread, and here https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/2 as well. The other part being that it allows explicitly marking with overloads which types of type expressions functions handle

@davidfstr

@davidfstr I think this discussion should be happening on discourse as it affects more than just mypy.

Agreed. I have requested that folks respond on the discourse thread, but most responses have actually happened here so far. I have been giving counter-responses in the same venue that I received responses, which so far has been mostly here.

As for the concern Eric brought up, that was (part of) the reason for what I mentioned with TypeForm[T, *Args] earlier in this thread

I'm not sure how

the TypeForm[OriginType, *ArgTypes] syntax

is related to

Eric's concern that it would be difficult for a static type checker like pyright to match TypeExpressions[]s in locations that would normally accept only a regular value expression.

[the TypeForm[OriginType, *ArgTypes] syntax] allows explicitly marking with overloads which types of type expressions functions handle

Yes, however I've already mentioned that: I’m not sure that its desirable (or even possible) to write out the exact constraints on what kinds of forms a particular function will accept. (And I think that’s OK.) I also responded further later in the thread.

Draft 3 of the ~~TypeForm~~ TypeExpr PEP is ready for review. Please leave your comments in the discuss.python.org thread.

[PEP 747] Recognize TypeForm[T] type and values (python#9773)

EDIT:

Never mind, was looking at the latest https://github.com/davidfstr/mypy/tree/f/typeform3 , vs. the open PR which references an earlier commit. Apologies for the noise.

ORIGINAL:

Is it expected that old-style Union[X | Y] unions are not valid TypeForm objects, but new-style X | Y unions are?

One of the use-cases this PEP facilitates is proper type hinting for runtime object creation, e.g.:

# contrived example for simplicity - but think of things like Pydantic, typedload, etc
def make_obj[T](type_: TypeForm[T], *args, **kwargs) -> T:
    for t_ in (type_, *get_args(type_)):
        try:
            return t_(*args, **kwargs)
        except:
            pass
    raise ValueError

X | Y unions work:

x = make_obj(int | str, "s") # OK
reveal_type(x) # Revealed type is "Union[builtins.int, builtins.str]"

but Union[X | Y] unions don't:

y = make_obj(Union[int, str], "s") # error: Argument 1 to "make_obj" has incompatible type "<typing special form>"; expected "TypeForm[Never]"  [arg-type]
reveal_type(y) # Revealed type is "Any"

I know there are historical differences between typing.Union and types.UnionType which might make this a nontrivial problem to solve, but it's still confusing behaviour. A lot of the utility of TypeForm is for libraries performing runtime introspection of type annotation objects, and library maintainers can't control whether client code is using old-vs-new style unions. End users are explicitly taught to consider both forms as identical (e.g. in the docs for typing.Union - or, ironically, in the above reveal_type example where Mypy itself describes the int | str object as Union[int, str]).

TypeForm[T] generics are a great concept, particularly with unions, but the current state leaves me uncertain whether unions are even meant to be supported.

@davidfstr

That seems wrong; Union[int, str] should be accepted. Are you trying this on @davidfstr's PR branch?

That seems wrong; Union[int, str] should be accepted. Are you trying this on @davidfstr's PR branch?

Hmm, the original source branch the PR was created from ( https://github.com/davidfstr/mypy/tree/f/typeform3 ) doesn't accept Union[int, str] but I see now the PR here references an older commit from his fork, and that does work. I know discussion on this idea has evolved over time so I assumed his fork was the "current state", but chalk it up to my mistake. Apologies for the noise.

[PEP 747] Recognize TypeForm[T] type and values (python#9773)

[PEP 747] Recognize TypeForm[T] type and values (#9773) (#19596)

Navigation Menu

Repository navigation

TypeForm[T]: Spelling for regular types (int, str) & special forms (Union[int, str], Literal['foo'], etc) #9773

92 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

TypeForm[T]: Spelling for regular types (int, str) & special forms (Union[int, str], Literal['foo'], etc) #9773

Description

Activity

hauntsaninja commented on Dec 1, 2020

JelleZijlstra commented on Dec 1, 2020

davidfstr commented on Dec 1, 2020

davidfstr commented on Dec 1, 2020

gvanrossum commented on Dec 4, 2020

ltworf commented on Dec 4, 2020

davidfstr commented on Dec 5, 2020

davidfstr commented on Dec 5, 2020

gvanrossum commented on Dec 5, 2020

davidfstr commented on Dec 8, 2020

gvanrossum commented on Dec 8, 2020

davidfstr commented on Dec 8, 2020

davidfstr commented on Dec 14, 2020

92 remaining items

JelleZijlstra commented on May 10, 2024

mikeshardmind commented on May 11, 2024

davidfstr commented on May 12, 2024

davidfstr commented on Jun 17, 2024

dchevell commented on Jun 26, 2025

JelleZijlstra commented on Jun 26, 2025

dchevell commented on Jun 26, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions