Skip to content

Add typing.get_protocol_members and typing.is_protocol #104873

Description

@JelleZijlstra

#103160 added an attribute __protocol_attrs__ that holds the names of all protocol attributes:

>>> from typing import Protocol
>>> class X(Protocol):
...     a: int
...     def b(self) -> str: pass
... 
>>> X.__protocol_attrs__
{'b', 'a'}

This is useful for code that needs to extract the names included in a Protocol at runtime. Previously, this was quite difficult, as you had to look at the class's __dict__ and __annotations__ directly and exclude a long list of internal attributes (e.g. https://github.com/quora/pyanalyze/blob/bd7f520adc2d8b098be657dfa514d1433bea3b0c/pyanalyze/checker.py#L428).

However, currently __protocol_attrs__ is an undocumented private attribute. I think we should either document it or add an introspection helper like typing.get_protocol_attrs() that exposes it.

Linked PRs

Activity

  1. AlexWaygood commented on May 24, 2023

    @AlexWaygood
    Member

    The worry about documenting it is that people might start monkey-patching it, which I don't want to encourage :)

    We could mitigate that by making it a read-only property (and by making it a frozenset instead of a mutable set). But making it a property might introduce some performance overhead, and the whole reason we added this attribute was to address a performance issue. Having a dunder attribute also wouldn't work very well with static type checkers -- because of how special Protocol is in typeshed's stubs, we wouldn't be able to add a stub for the attribute in typeshed, so mypy et al. would complain every time a user tried to access the dunder on a protocol class.

    So, I guess I vote for a typing.get_protocol_attrs() "getter" function! It feels like it avoids all the problems I mentioned above.

  2. JelleZijlstra commented on May 24, 2023

    @JelleZijlstra
    MemberAuthor

    Yes, I think a getter function makes sense here. I was just thinking about this while writing a patch for #104874, where I do think it makes sense to simply document the dunder attribute. The difference there is that the NewType class itself is simple and the attribute isn't used at runtime. Protocols, on the other hand, have to deal with complex subclassing relationships and it's plausible we'll want to make changes to the implementation in the future that change how the dunder works.

  3. AlexWaygood commented on May 24, 2023

    @AlexWaygood
    Member

    Yes, I think a getter function makes sense here. I was just thinking about this while writing a patch for #104874, where I do think it makes sense to simply document the dunder attribute. The difference there is that the NewType class itself is simple and the attribute isn't used at runtime. Protocols, on the other hand, have to deal with complex subclassing relationships and it's plausible we'll want to make changes to the implementation in the future that change how the dunder works.

    Agreed on all counts.

  4. AlexWaygood commented on May 24, 2023

    @AlexWaygood
    Member

    get_protocol_members() would probably be a better name for the function, btw. I called the attribute __protocol_attrs__ because the attribute __protocol_attrs__ is calculated by calling typing._get_protocol_attrs (which has been in typing for many years, and there's no real reason to rename). But I think __protocol_members__ would probably have been a better name for the attribute, really. I probably would have thought longer about the name if I'd planned for it to become public API ;)

  5. added a commit that references this issue on May 24, 2023
  6. changed the title [-]Make `__protocol_attrs__` documented and public[/-] [+]Add `typing.get_protocol_members` and `typing.is_protocol`[/+] on May 24, 2023
  7. JelleZijlstra commented on May 24, 2023

    @JelleZijlstra
    MemberAuthor

    Based on PR review, going to also add typing.is_protocol.

  8. Giddius commented on May 25, 2023

    @Giddius

    Based on PR review, going to also add typing.is_protocol.

    I know that this is most relevant for typing but just as comment and not actual critique:
    Almost all is_x functions that are not builtins are part of inspect and most do not use the underscore format:

    • inspect.isabstract
    • inspect.isawaitable
    • inspect.isdatadescriptor
    • inspect.iscoroutine
    • …

    With asyncio.iscoroutine being in asyncio and typing.is_typeddict in typing being the exceptions.

    ——
    This again is meant more as a comment not a critic or request.

    Edit: removed word at the end that was left in by accident.

  9. added a commit that references this issue on Jun 14, 2023
  10. AlexWaygood commented on Jun 20, 2023

    @AlexWaygood
    Member

    Oh, I think this is done now! 🎉

  11. NeilGirdhar commented on Jul 2, 2023

    @NeilGirdhar

    Just out of curiosity, but does is_protocol(x) differ from issubclass(x, Protocol) and x not in {typing_extensions.Protocol, typing.Protocol}?

  12. AlexWaygood commented on Jul 2, 2023

    @AlexWaygood
    Member

    Just out of curiosity, but does is_protocol(x) differ from issubclass(x, Protocol) and x not in {typing_extensions.Protocol, typing.Protocol}?

    Yes — x might be a subclass of typing_extensions.Protocol, and typing.is_protocol(x) should still evaluate to True

  13. AlexWaygood commented on Jul 2, 2023

    @AlexWaygood
    Member

    @NeilGirdhar the relevant commit is here if you'd like to read the source code :-) fc8037d

  14. JelleZijlstra commented on Jul 2, 2023

    @JelleZijlstra
    MemberAuthor

    The more important reason is that concrete classes can be subclasses of Protocol.

    In [1]: from typing import Protocol
    
    In [2]: class Proto(Protocol):
       ...:     def f(self) -> int: ...
       ...: 
    
    In [3]: class Concrete(Proto):
       ...:     def f(self) -> int: return 0
       ...: 
    
    In [4]: issubclass(Concrete, Protocol)
    Out[4]: True
    
  15. NeilGirdhar commented on Jul 2, 2023

    @NeilGirdhar

    @JelleZijlstra Makes perfect sense, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions