Repository navigation
types.UnionType is not subscriptable #89581
Description
Activity
wyfo commented
on Oct 9, 2021 wyfomannequinMannequinAuthortypes.UnionTypeis not subscriptable, and this is an issue when type manipulations are done.A common maniputation I've to do is to substitute all the
TypeVarof a potential generic type by their specialization in the given context.
For example, given a class:@dataclass class Foo(Generic[T]): bar: list[T] baz: T | None
in the case of
Foo[int], I want to compute the effective type of the fields, which will belist[int]andint | None.
It could be done pretty easily by a recursive function:def substitute(tp, type_vars: dict): origin, args = get_origin(tp), get_args(tp) if isinstance(tp, TypeVar): return type_vars.get(tp, tp) elif origin is Annotated: return Annotated[(substitute(args[0], type_vars), *args[1:])] else: return origin[tuple(substitute(arg) for arg in args)] # this line fails for types.UnionType
And this is not the only manipulation I've to do on generic types. In fact, all my library (apischema) is broken in Python 3.10 because of
types.UnionType.
I've to substitutetypes.UnionTypebytyping.Unioneverywhere to make things work;types.UnionTypeis just not usable for dynamic manipulations.I've read PEP-604 and it doesn't mention if
types.UnionTypeshould be subscriptable or not. Is there a reason for not making it subscriptable?I don't understand your example, T | None doesn't return a types.Union object, it returns typing.Union/typing.Optional. (I'm assuming this T is the TypeVar in typing). Which *is* subscriptable.
>>> (T | None)[int].__origin__ typing.Union
If you meant to say: why is typing.Union[] allowed, but not types.UnionType[]? That is intentional. types.UnionType is only meant for builtin types. Once you union with *any* type from typing, it will convert to a typing.Union.
>>> type(int | str) <class 'types.UnionType'> >>> int | str | T typing.Union[int, str, ~T]
If you intend to reconstruct a types.Union from another types.Union, you can do:
args = get_args(int | str) import operator, functools functools.reduce(operator.or_, args)
And guard this code with an isinstance(tp, types.UnionType) check.
wyfo commented
on Oct 10, 2021 wyfomannequinMannequinAuthorIndeed, sorry, my example was bad. My library was raising at several place, and I've extrapolated about generic substitution.
I've indeed other substitutions (without
TypeVar), and because they were failing, I've assumed that all of my substitutions were failing; I was wrong about generic one.For example, if I want to substitute
int | Collection[int]toint | list[int], I will have to replacetypes.UnionTypebytyping.Unionor usereduce, while it was not necessary in 3.9 where I could just writeget_origin(tp)[new_args].So I'll have to add some
ifin my code.No worries!
So I'll have to add some
ifin my code.Yeah, we had to do that in the typing module too. Hope you manage to fix your library without much trouble.
If you meant to say: why is typing.Union[] allowed, but not
types.UnionType[]? That is intentional. types.UnionType is only meant for
builtin types. Once you union with *any* type from typing, it will convert
to a typing.Union.But why? Just so types.UnionType (if it has a typevar) doesn’t have to
support subscriptions? Even if this saves us now, I agree with OP that it
ought to allow it, so we can deprecate typing.Union properly. And e.g.
dict[str, T] works.--Guido (mobile)
I hope I didn't misunderstand you, but to clarify, what OP is asking is an alternative way to construct types.UnionType objects and write:
types.UnionType[int, str]
like how we used to write before 3.10:
typing.Union[int, str]
I don't know why we need this. We can write
int | str. The reason for PEP-604 in the first place was to avoid the subscript syntax and use|since it's cleaner. OP's use case is for reconstructing types.UnionType objects easily, butfunctools.reduce(operator.or_, args)works.Re: TypeVar subscription; PEP-604 syntax already supports that. We used to implement that in C. After Serhiy's Great Cleanup, a bitwise OR with a TypeVar automatically converts types.UnionType to typing.Union. So all the TypeVar support is now done in Python.
>>> type(int | str) <class 'types.UnionType'> >>> (int | str | T)[dict] typing.Union[int, str, dict]
Oh, I see. No, they must make another special case, like for Annotated.--
--Guido (mobile)types.UnionType corresponds to typing._UnionGenericAlias, not typing.Union.
We can make (int | str | T)[dict] returning an instance of types.UnionType instead of an instance of typing._UnionGenericAlias. But it will be a breaking change, because typing._UnionGenericAlias and types.UnionType are different and not completely compatible types. We should wait some time before making such changes, so all user code will be made supporting both typing._UnionGenericAlias and types.UnionType.
If the user code does something special like substituting
int | Collection[int]toint | list[int], it should have some additional ifs in any case, otherwise it will not recognize new typing types including types.UnionTypes. And subscription does not work in all typing types, we have copy_with() for some types and special cases for others in the code of the typing module. I am going to unify it finally, but it takes time, my time and user's time to migrate to new idioms.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: