Skip to content

Improve the constructors of AST nodes #105858

Closed
Closed
Improve the constructors of AST nodes#105858
Labels
stdlibStandard Library Python modules in the Lib/ directorytopic-parsertype-featureA feature request or enhancement

Description

@JelleZijlstra
JelleZijlstra
opened on Jun 16, 2023

Currently, the constructors for AST nodes accept arbitrary keyword arguments and don't enforce any value:

>>> node=ast.FunctionDef(what="is this")
>>> node.what
'is this'
>>> node.name
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'FunctionDef' object has no attribute 'name'

Problems with the current situation:

  • To make a correct AST node, you have to pass every attribute, including ones that could sensibly default to an empty list
  • You cannot rely on attributes like name being present
  • It's possible to create useless, broken AST nodes like the above
  • Introspection tools can't easily tell what the signature of the constructor is supposed to be
  • Adding new fields to an AST node causes various compatibility issues (PEP-695: Potentially breaking changes made to __match_args__ attributes of AST nodes #104799)

Proposed solution for 3.13:

  • Default all fields to some sensible value if they are not provided to the constructor: an empty list for list fields, and None for other fields.
  • Emit a DeprecationWarning if the constructor is passed arguments that it doesn't recognize (e.g., what above). In 3.15, this will raise an error.
  • Add a __text_signature__ to the AST classes indicating the expected signature.

Linked PRs

Activity

  1. JelleZijlstra
    added
    type-featureA feature request or enhancement
    topic-parser
    on Jun 16, 2023

  2. furkanonder commented on Jun 16, 2023

    @furkanonder
    furkanonder
    on Jun 16, 2023
    Contributor

    CC: @isidentical

  3. JelleZijlstra
    added a commit that references this issue on Jun 17, 2023

    pythongh-105858: Improve AST node constructors

    b1b73d9

  4. brandtbucher commented on Jun 20, 2023

    @brandtbucher
    brandtbucher
    on Jun 20, 2023
    Member

    Yes please! This has been such a pain point for me.

    One note: we should probably only default optional fields (?) to None, and require everything else.

  5. JelleZijlstra commented on Jun 20, 2023

    @JelleZijlstra
    JelleZijlstra
    on Jun 20, 2023
    MemberAuthor

    One note: we should probably only default optional fields (?) to None, and require everything else.

    Agree, I ended up doing that in my draft PR. Omitting a required field (e.g. FunctionDef.name) will raise a DeprecationWarning, as will passing a bogus field.

  6. iritkatriel
    added
    stdlibStandard Library Python modules in the Lib/ directory
    on Nov 27, 2023
  7. JelleZijlstra
    added a commit that references this issue on Feb 28, 2024

    gh-105858: Improve AST node constructors (#105880)

    ed4dfd8

  8. JelleZijlstra commented on Feb 28, 2024

    @JelleZijlstra
    JelleZijlstra
    on Feb 28, 2024
    MemberAuthor

    Merged!

  9. JelleZijlstra
    closed this as completedon Feb 28, 2024
  10. Privat33r-dev
    added a commit that references this issue on Feb 28, 2024

    pythongh-105858: Improve AST node constructors (python#105880)

    1745629

  11. JelleZijlstra commented on Feb 28, 2024

    @JelleZijlstra
    JelleZijlstra
    on Feb 28, 2024
    MemberAuthor

    Reopening as I broke all the buildbots

  12. JelleZijlstra
    reopened this on Feb 28, 2024
  13. JelleZijlstra
    added a commit that references this issue on Feb 28, 2024

    pythongh-105858: Expose some union-related objects as internal APIs

    5142722
  14. gvanrossum
    added a commit that references this issue on Feb 28, 2024

    pythongh-105858: Improve AST node constructors (python#105880)

    ab5339f
  15. encukou
    added a commit that references this issue on Feb 28, 2024

    gh-105858: Expose some union-related objects as internal APIs (GH-116025

    d53560d
  16. JelleZijlstra
    closed this as completedon Feb 28, 2024
  17. woodruffw
    added 2 commits that reference this issue on Mar 4, 2024

    pythongh-105858: Improve AST node constructors (python#105880)

    7ddc0d8

    pythongh-105858: Expose some union-related objects as internal APIs (p…

    a5fd325
  18. adorilson
    added 2 commits that reference this issue on Mar 25, 2024

    pythongh-105858: Improve AST node constructors (python#105880)

    052e900

    pythongh-105858: Expose some union-related objects as internal APIs (p…

    7eb8096

  19. hroncok commented on Mar 26, 2024

    @hroncok
    hroncok
    on Mar 26, 2024
    Contributor

    There might be a regression: #117266

  20. diegorusso
    added 2 commits that reference this issue on Apr 17, 2024

    pythongh-105858: Improve AST node constructors (python#105880)

    3a32841

    pythongh-105858: Expose some union-related objects as internal APIs (p…

    e7f3eda
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytopic-parsertype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions