Skip to content

Implement CALL_FUNCTION adaptive interpreter optimizations #88691

Closed
Closed
Implement CALL_FUNCTION adaptive interpreter optimizations#88691
Labels
3.11only security fixespendingThe issue will be closed if no feedback is providedperformancePerformance or resource usage

Description

@Fidget-Spinner
Fidget-Spinner
opened on Jun 28, 2021
BPO 44525
Nosy @markshannon, @pablogsal, @sobolevn, @Fidget-Spinner, @thesamesam
PRs
  • bpo-44525: Specialize CALL_FUNCTION for C function calls #26934
  • bpo-44525: Specialize simple Python calls. #29033
  • bpo-44525: Add recursive checks for CALL_FUNCTION_BUILTIN_O #29271
  • bpo-44525: Copy freevars in bytecode to allow calls to inner functions to be specialized #29595
  • bpo-44525: Specialize for calls to type and other builtin classes with 1 argument. #29942
  • bpo-44525: Split calls into PRECALL and CALL #30011
  • bpo-44525: Document new call opcodes for 3.11 #30107
  • bpo-44525: Specialize calls to normal python classes #30415
  • bpo-46709: fix race conditions in unittest/test_brake #31273
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2021-06-28.14:22:18.192>
labels = ['3.11', 'performance']
title = 'Implement CALL_FUNCTION adaptive interpreter optimizations'
updated_at = <Date 2022-02-11.11:31:07.338>
user = 'https://github.com/Fidget-Spinner'

    bugs.python.org fields:

    activity = <Date 2022-02-11.11:31:07.338>
actor = 'sobolevn'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-06-28.14:22:18.192>
creator = 'kj'
dependencies = []
files = []
hgrepos = []
issue_num = 44525
keywords = ['patch']
message_count = 9.0
messages = ['396639', '404376', '404470', '405108', '405192', '406831', '408554', '408609', '409779']
nosy_count = 5.0
nosy_names = ['Mark.Shannon', 'pablogsal', 'sobolevn', 'kj', 'thesamesam']
pr_nums = ['26934', '29033', '29271', '29595', '29942', '30011', '30107', '30415', '31273']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue44525'
versions = ['Python 3.11']

    Linked PRs

    Activity

    1. Fidget-Spinner commented on Jun 28, 2021

      @Fidget-Spinner
      Fidget-Spinner
      on Jun 28, 2021
      MemberAuthor

      CALL_FUNCTION can be specialized. Copying from Mark's comments faster-cpython/ideas#54 (comment)

      There are a number of specializations of CALL_FUNCTION that make sense:

1. Calls to a Python function where the arguments and parameters match exactly.
2. Calls to a Python function with keyword arguments, or defaults, when the argument shuffle can be pre-computed.
3. Calls to builtin functions.
4. Calls to specific builtin functions, specifically len and instance.
5. Calls to type with a single argument.
6. Instantiation of a "normal" class, that is a class that does not redefine __new__ and whose metaclass is type.

      I've prepared a PR that should speed up non-keyword calls to PyCFunction. It covers specializations 3 and 4. Stable pyperf microbenchmarks show 5-15% less call overhead for some PyCFunctions. Please see faster-cpython/ideas#54 (comment) for the benchmark script and results.

      This issue is also tied to bpo-44207 (Add a version number to Python functions) which will be required for specializations 1 and 2.

    2. Fidget-Spinner
      added
      3.11only security fixes
      performancePerformance or resource usage
      on Jun 28, 2021

    3. markshannon commented on Oct 19, 2021

      @markshannon
      markshannon
      on Oct 19, 2021
      Member

      New changeset 3163e68 by Ken Jin in branch 'main':
      bpo-44525: Specialize CALL_FUNCTION for C function calls (GH-26934)
      3163e68

    4. markshannon commented on Oct 20, 2021

      @markshannon
      markshannon
      on Oct 20, 2021
      Member

      New changeset 8863a0f by Mark Shannon in branch 'main':
      bpo-44525: Specialize simple Python calls. (GH-29033)
      8863a0f

    5. pablogsal commented on Oct 27, 2021

      @pablogsal
      pablogsal
      on Oct 27, 2021
      Member

      Unfortunately, PR 26934 has broken thes 390x RHEL7 LTO 3.x buildbot as you can see before. As per the buildbot maintenance procedures, we will need to revert this PR unless is fixed in 24 hours.

      @markshannon @Fidget-Spinner

    6. markshannon commented on Oct 28, 2021

      @markshannon
      markshannon
      on Oct 28, 2021
      Member

      New changeset 0a1a36b by Ken Jin in branch 'main':
      bpo-44525: Add recursive checks for CALL_FUNCTION_BUILTIN_O (GH-29271)
      0a1a36b

    7. markshannon commented on Nov 23, 2021

      @markshannon
      markshannon
      on Nov 23, 2021
      Member

      New changeset 135cabd by Mark Shannon in branch 'main':
      bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
      135cabd

    8. markshannon commented on Dec 14, 2021

      @markshannon
      markshannon
      on Dec 14, 2021
      Member

      New changeset 9f8f451 by Mark Shannon in branch 'main':
      bpo-44525: Split calls into PRECALL and CALL (GH-30011)
      9f8f451

    9. markshannon commented on Dec 15, 2021

      @markshannon
      markshannon
      on Dec 15, 2021
      Member

      New changeset 3a60bfe by Mark Shannon in branch 'main':
      bpo-44525: Specialize for calls to type and other builtin classes with 1 argument. (GH-29942)
      3a60bfe

    10. markshannon commented on Jan 5, 2022

      @markshannon
      markshannon
      on Jan 5, 2022
      Member

      See faster-cpython/ideas#210

    11. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    12. iritkatriel commented on Sep 9, 2022

      @iritkatriel
      iritkatriel
      on Sep 9, 2022
      Member

      Is there anything left to do on this?

    13. iritkatriel
      added
      pendingThe issue will be closed if no feedback is provided
      on Sep 9, 2022

    14. pablogsal commented on Sep 10, 2022

      @pablogsal
      pablogsal
      on Sep 10, 2022
      Member

      Closing. Please feel free to re-open if we missed anything.

    15. pablogsal
      closed this as completedon Sep 10, 2022
    16. brandtbucher
      added a commit that references this issue on Apr 5, 2023

      GH-88691: Shrink the CALL caches (GH-103230)

      b4978ff
    17. gaogaotiantian
      added a commit that references this issue on Apr 8, 2023

      pythonGH-88691: Shrink the CALL caches (pythonGH-103230)

      0b43d39
    18. warsaw
      added a commit that references this issue on Apr 11, 2023

      pythonGH-88691: Shrink the CALL caches (pythonGH-103230)

      a568403
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.11only security fixespendingThe issue will be closed if no feedback is providedperformancePerformance or resource usage

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions