__name__ attribute in typing module #88690

I noticed some (perhaps intentional) oddities with the __name__ attribute:

• typing classes like Any (subclass of _SpecialForm) do not have a __name__ attribute,
• abstract base classes in typing, like MutableSet do not have a __name__ attribute,
• 'ChainMap', 'Counter', 'OrderedDict' do not have a __name__ attribute when imported from typing, but do when imported from collections.

I have written a function to show presence/absence if the name __name__ attribute:

def split_module_names(module):
    unnamed, named = set(), set()
    for name in dir(module):
        if not name.startswith('_'):
            attr = getattr(module, name)
            try:
                if hasattr(attr, '__name__'):
                    named.add(name)
                else:
                    unnamed.add(name)
            except TypeError:
                pass
    return named, unnamed

import typing
import collections

typing_named, typing_unnamed = split_module_names(typing)
collec_named, collec_unnamed = split_module_names(collections)

print("typing_unnamed:", typing_unnamed)
print("collec_named & typing_unnamed:", collec_named & typing_unnamed)

Is this intentional? It seems a little inconsistent.

I also found something that sometimes the __name__ attribute does resolve:

class S(typing.Sized):
        def __len__(self):
            return 0

print("'Sized' in typing_unnamed:", 'Sized' in typing_unnamed)
print("[t.__name__ for t in S.__mro__]:", [t.__name__ for t in S.__mro__])  # here __name__ is resolved!
print("getattr(typing.Sized, '__name__', None):", getattr(typing.Sized, '__name__', None))

printing:

'Sized' in typing_unnamed: True
[t.__name__ for t in S.__mro__]: ['S', 'Sized', 'Generic', 'object']
getattr(typing.Sized, '__name__', None): None

Is this intentional? It seems a little inconsistent.

The __name__ attribute is for internal use only. It's subject to change every release along with other implementation details.

Sorry, I don't really understand what this issue is requesting. Do you want to add the split_module_names function or standardize __name__ or something else? Depending on what you're suggesting the follow up would be different.

I was not aware the __name__ attribute is an implementation detail. It is described in the docs: https://docs.python.org/3/reference/datamodel.html.

I have been using it since python 2.7, for example for logging.

The function “split_module_names” is just a function to see what items in a module have and do not have a __name__ attribute; thought it might help proceedings.

If I were to suggest an improvement, it would be that all classes and types (or minimally the abc’s) would have a __name__ attribute, being the name under which it can be imported.
Also that the abc’s in typing and collections are as similar as possible.

Lars, yes you're right that __name__ is documented in datamodel, sorry I wasn't clear in my original message. What I meant was that specifically for the typing module, it's not exposed anywhere in its docs https://docs.python.org/3/library/typing.html.

If I were to suggest an improvement, it would be that all classes and types (or minimally the abc’s) would have a __name__ attribute, being the name under which it can be imported.

I think this makes sense. It should be as simple as adding self.__name__ = name or some variant. Note that some types hack their names, such as TypeVar or ParamSpec. So it's not always that __name__ ​== type/class name.

Also that the abc’s in typing and collections are as similar as possible.
We strive towards this but it's difficult to get it 100%. The abcs in typing are implemented in pure Python and alias the ones in collections, while the ones in collections are sometimes tied to C. AFAIK, most types in typing only do what the PEPs promise. I hope you understand.

It sounds reasonable to add the __name__ attribute. Since these objects
aren't really types, the default mechanism for constructing a type doesn't
give them this. Are there other attributes that are missing? We should
probably add those too.

I have been doing some research, but note that I don't have much experience with the typing module. That said, there seem to be 2 main cases:

• '_SpecialForm': with instances Any, Union, etc.
• '_BaseGenericAlias'/'_SpecialGenericAlias': base classes collections classes.

I think '_SpecialForm' can be enhanced to have '__name__' by replacing the '_name' attribute with '__name__'. Maybe add '__qualname__' as well. I cannot say whether there are many more attributes that could be implemented to have the same meaning as in 'type'. The meaning of attributes like '__mro__' seem difficult to define.
Alternatively '__getattr__' could be added (but that might be too much):

def __getattr__(self, attr):
    return getattr(self._getitem, attr)

'_BaseGenericAlias''_SpecialGenericAlias' the '__getattr__' method could perhaps be adapted (or overridden in '_SpecialGenericAlias') as follows, from:

def __getattr__(self, attr):
    # We are careful for copy and pickle.
    # Also for simplicity we just don't relay all dunder names
    if '__origin__' in self.__dict__ and not _is_dunder(attr):
        return getattr(self.__origin__, attr)
    raise AttributeError(attr)

to:

def __getattr__(self, attr):
    if '__origin__' in self.__dict__:
        return getattr(self.__origin__, attr)
    raise AttributeError(attr)

or perhaps:

def __getattr__(self, attr):
    if '__origin__' in self.__dict__ and hasattr(type, attr):
        return getattr(self.__origin__, attr)
    raise AttributeError(attr)

to forward unresolved attribute names to the original class.

I have written some tools and tested some with the above solutions and this seems to solve the missing '__name__' issue and make the typing abc's much more in line with the collection abc's. However I did not do any unit/regression testing (pull the repo, etc.)

tools are attached.

Sorry for the slow progress. I don’t think it is important for Any orUnion to have these attributes, but the ones that match ABCs or concrete classes (e.g. MutableSet, Counter) should probably have __name__, __qualname__, and __module__, since the originals have those. I think __module__ should be set to ‘typing’, and __qualname__ to ‘typing.WhatEver’.

Happy to see progress on this issue and I can see that adding these attributes to the ABC's in typing makes the most sense. However for my direct use-case (simplified: using Any in a type checking descriptor) it would be really practical to have the __name__ (and perhaps __qualname__ and __module__) attributes in the Any type. This is mainly for consistent logging/printing purposes.

Since Any already has a _name attribute, changing this to __name__ might achieve this.

I think __module__ should be set to ‘typing’, and __qualname__ to ‘typing.WhatEver’.

PEP-3155 specifies that __qualname__ does not include the module name:
https://www.python.org/dev/peps/pep-3155/#excluding-the-module-name

Rather, it's for nested classes and classes created in local scopes.

Yurii has a working PR for __name__ in _BaseGenericAlias, but not for _SpecialForm yet.

Guido and/or Lukasz, do y'all think we should support __name__ and __qualname__ for special forms too? Personally I don't see how it'd hurt and I'm +1 for this.

I see this as part of a trend to improve runtime introspection of complex
type expressions. That seems to be going ahead regardless of whether we
like it or not, so let's do this.

New changeset a3a4d20 by Yurii Karabas in branch 'main':
bpo-44524: Fix cryptic TypeError message when trying to subclass special forms in typing (GH-27710)
a3a4d20

There are still cryptic TypeError messages for Annotated:

>>> class X(Annotated[int | float, "const"]): pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

There are some side effects of setting _name. In 3.9:

>>> class X(Annotated[int, (1, 10)]): pass
... 
>>> X.__mro__
(<class '__main__.X'>, <class 'int'>, <class 'object'>)

In 3.10:

>>> class X(Annotated[int, (1, 10)]): pass
... 
>>> X.__mro__
(<class '__main__.X'>, <class 'int'>, <class 'typing.Generic'>, <class 'object'>)

Now a subclass of an Annotated alias is a generic type. Should it be?

Now a subclass of an Annotated alias is a generic type. Should it be?

I'm unsure if Annotated should be subclassable in the first place, but if I understand PEP-593 correctly,
class X(Annotated[int, (1, 10)]), should be equivalent to class X(int) right? If that's the case, it's subclassable and Generic shouldn't be in the MRO.

FWIW, the other special forms don't allow subclassing, so we don't need to think about this problem for them. Annotated is a special cookie.

I propose we just drop the _name hack temporarily in Annotated. A real fix requires fixing up __mro_entries__, but I am uncomfortable with us backporting to 3.10 anything that touches __mro_entries__ due to the numerous edge cases it has and how close we are to 3.10 final.

I don't think we need to support Annotated as a base class. PEP-593 is titled "Flexible function and variable annotations", and base classes are neither of those things. None of the examples in the PEP or the implementation use Annotated as a base class either.

On the other hand, subclassing Annotated[T, ...] does work at runtime in 3.9, so maybe we're bound by backward compatibility now.

New changeset 4ceec49 by Serhiy Storchaka in branch 'main':
bpo-44524: Do not set _name of _SpecialForm without need (GH-27861)
4ceec49

New changeset 5bd27c3 by Miss Islington (bot) in branch '3.10':
bpo-44524: Do not set _name of _SpecialForm without need (GH-27861) (GH-27871)
5bd27c3

New changeset 23384a1 by Ken Jin in branch 'main':
bpo-44524: Don't modify MRO when inheriting from typing.Annotated (GH-27841)
23384a1

New changeset 06e9a35 by Miss Islington (bot) in branch '3.10':
bpo-44524: Don't modify MRO when inheriting from typing.Annotated (GH-27841)
06e9a35

New changeset 81fa08c by Miss Islington (bot) in branch '3.10':
bpo-44524: Fix cryptic TypeError message when trying to subclass special forms in typing (GH-27710)
81fa08c

Is there anything left to do here, or can this now be closed?

This issue has gone through a bit of a journey, but the original complaint was that __name__ was missing on typing objects. I checked @farcat's script on current main and the only objects in typing without a __name__ are {'EXCLUDED_ATTRIBUTES', 'TYPE_CHECKING'}, which seems reasonable enough.