Repository navigation
typing.Union does not support attribute assignment post gh-105511 #132139
Description
Activity
- changed the title
[-][3.14 Regression] `Union` variable does not allow attribute assignment anymore[/-][+][3.14 Regression] `typing.Union` variable does not allow attribute assignment anymore[/+]on Apr 5, 2025 Was it designed to support custom attributes? is it a documented behavior? (I don't know so I'm just asking if this was deliberate)
Yeah, don't do that. Setting arbitrary attributes on Union objects (or any other typing objects) was never supported and if you do that sort of thing, be prepared for your code to break.
That said, we could allow setting arbitrary attributes again by adding a managed
__dict__to union objects, at a cost of increasing memory usage for every other user of unions. I'd be curious to hear opinions on whether that's worth it.at a cost of increasing memory usage for every other user of unions
How much of a cost are we talking about? I think you mentioned something about the fact that unions' size is smaller (or was it larger?) but I can't find the issue (I think I read this yesterday somewhere). If we're only talking about a few bytes when it's not used, then why not. But if we're doubling the union sizes I think it's maybe not worth.
Now, are there other projects for which this could be needed? if there is enough usage, and from large libraries, then maybe it's a possibility? By the way, for what reason do you actually need unions to support assignment?
- changed the title
[-][3.14 Regression] `typing.Union` variable does not allow attribute assignment anymore[/-][+]`typing.Union` variable does not allow attribute assignment anymore post gh-105511[/+]on Apr 5, 2025 - changed the title
[-]`typing.Union` variable does not allow attribute assignment anymore post gh-105511[/-][+]`typing.Union` variable does not allow attribute assignment post gh-105511[/+]on Apr 5, 2025 - changed the title
[-]`typing.Union` variable does not allow attribute assignment post gh-105511[/-][+]`typing.Union` does not support attribute assignment post gh-105511[/+]on Apr 5, 2025 11 remaining items
- added a commit that references this issue
on Apr 6, 2025 - added a commit that references this issue
on Apr 6, 2025 Another interesting side effect of this is for "public API specifications".
Systems like sphinx considers an API public in the current file if the API's__module__attribute is set to the current file.We use this in PyTorch in particular to define types, make them public API from sphinx point of view to properly document. An example is here: https://github.com/pytorch/pytorch/blob/aacb9440791f4e9567e2123f19bb062e75c2d242/torch/ao/quantization/__init__.py#L34-L36
The logic at
makes the behavior a bit surprising as accessing theLines 400 to 403 in c7d24b8
__module__attribute works but trying to set it fails with'typing.Union' object has no attribute '__module__' and no __dict__ for setting new attributes(the attribute actually does exist).Curious to hear what you think here?
Are we misguided on trying to set the__module__attribute as a marker of public APIs for the Union object? Or we should try and find a way to enable this behavior?We use this in PyTorch in particular to define types, make them public API from sphinx point of view to properly document.
@albanD If it is only related to the docs build, I have worked around this by adding a custom
typehint_formmatterand saving theUnionobjects in a weakrefset:def typehints_formatter(annotation, config=None): if 'optree' not in sys.modules: sys.path.insert(0, str(PROJECT_ROOT)) import optree if ( isinstance(annotation, type(typing.Union[int, str])) and typing.get_origin(annotation) is typing.Union and annotation in optree.PyTree.__instances__ ): param, name = optree.PyTree.__instances__[annotation] if name is not None: return f':py:class:`{name}`' from sphinx_autodoc_typehints import format_annotation return rf':py:class:`PyTree` \[{format_annotation(param, config=config)}]' return None
The keys of the
WeakKeyDictionaryare the set ofUnioninstances.https://github.com/metaopt/optree/blob/ebd1d78d5780b9ad8dbbaba89ecedfee3061d84d/optree/typing.py#L205-L211
https://github.com/metaopt/optree/blob/ebd1d78d5780b9ad8dbbaba89ecedfee3061d84d/optree/typing.py#L273-L275The logic at
Lines 400 to 403 in c7d24b8
static const char* const cls_attrs[] = {
"module", // Required for compatibility with typing module
NULL,
};
makes the behavior a bit surprising as accessing the__module__attribute works but trying to set it fails with'typing.Union' object has no attribute '__module__' and no __dict__ for setting new attributes(the attribute actually does exist).__module__is a class attribute. You cannot assign it via an instance.If it is only related to the docs build, I have worked around this by adding a custom typehint_formmatter and saving the Union objects in a weakrefset:
The problem is that it's not just that. We also have programmatic checks to ensure that things in the
__all__field matches with the__module__. But since I cannot change the__module__, I have to remove theUnionfrom the__all__field and thusimport *from that module changes behavior.Also given how it's defined you can't subclass Union to add a
__dict__to it either.- added a commit that references this issue
on Jul 15, 2025 To recapitulate why I am not interested in changing this:
- This was never supported or documented in any way.
- Reliance on this misfeature would have interacted poorly with the cache that existed before 3.14; if multiple modules each create say a
Union[int, str]and set its__module__, they would have clobbered each other. - This only worked for some kinds of types supported by the type system; for example, unions created through the
|syntax andtypes.GenericAliasdon't support attribute assignments; setting a__module__attribute on anAnnotatedobject sets its for allAnnotatedobjects. - Adding this now would require adding more memory usage to the union object, so all users would have to pay extra in memory usage.
As an alternative, I'd suggest using the
typestatement introduced in Python 3.12. Type aliases created through this feature automatically get the__module__they were created in.If you need to support 3.11 and earlier, you can use:
if sys.version_info >= (3, 12): exec("type Alias = A | B") else: Alias = Union[A, B] Alias.__module__ = __name__Reacted by Alex Waygood, albanD and Antonio SpadaroThanks for the in-depth answer. This all makes sense.
If you need to support 3.11 and earlier, you can use:
I'm afraid that doesn't work as the "type" statement will through an error in <=3.11 even if not executed:
$ cat foo.py import sys if sys.version_info >= (3, 12): type Alias = A | B else: Alias = Union[A, B] Alias.__module__ = __name__ $ python3.10 foo.py File "/home/albandes/local/pytorch/3.10_release_source/foo.py", line 4 type Alias = A | B ^^^^^ SyntaxError: invalid syntax $ python3.13 foo.py $
That is a great feature that would definitely solve our issue otherwise!
Sorry yes, you'd need to use exec. I was thinking about that but forgot to write it when I wrote the comment :). Fixed above.
Thanks for the quick answer @JelleZijlstra , given that exec makes our security people jumpy, I was trying to go with the following. Any concern with that approach?
if sys.version_info >= (3, 12): from typing import TypeAliasType Alias = TypeAliasType("Alias", A | B) else: Alias = Union[A, B] Alias.__module__ = __name__
Reacted by Jelle Zijlstra- added 2 commits that reference this issue
on Jul 27, 2025
Bug report
Bug description:
Before #105511,
typing.Unionis implemented in Python, and custom attributes can be assigned to aUnionvariable.However, after #105511,
typing.Unionis now implemented in C, and no attributes can be assigned to it. TheUnionTypedoes not allow subclassing; users cannot bypass this using subclassing.Repro:
./python -m pip install -v optree ./python -c 'import optree; optree.PyTree[int]'Source: https://github.com/metaopt/optree/blob/v0.15.0/optree/typing.py#L246-L254
cc @JelleZijlstra
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
Unionattributes #132157