Skip to content

Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call #90758

Closed
Closed
Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call#90758
Labels
3.11only security fixesbuildThe build process and cross-buildpendingThe issue will be closed if no feedback is providedperformancePerformance or resource usage

Description

@vstinner
vstinner
opened on Feb 1, 2022
BPO 46600
Nosy @vstinner, @methane, @corona10, @pablogsal
PRs
  • bpo-46600: ./configure --with-pydebug uses -Og with clang #31052
  • bpo-46600: Fix test_gdb.test_pycfunction() for clang -Og #31058
    		•
    Files
  • stack_overflow-4.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2022-02-01.13:06:00.875>
labels = ['3.11', 'build', 'performance']
title = 'Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call'
updated_at = <Date 2022-02-03.00:00:28.611>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2022-02-03.00:00:28.611>
actor = 'methane'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2022-02-01.13:06:00.875>
creator = 'vstinner'
dependencies = []
files = ['50600']
hgrepos = []
issue_num = 46600
keywords = ['patch']
message_count = 14.0
messages = ['412252', '412253', '412254', '412255', '412256', '412258', '412260', '412261', '412278', '412286', '412294', '412334', '412348', '412407']
nosy_count = 4.0
nosy_names = ['vstinner', 'methane', 'corona10', 'pablogsal']
pr_nums = ['31052', '31058']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue46600'
versions = ['Python 3.11']

    Activity

    1. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      Measure using this script on the main branch (commit 108e66b):
      ---

      import _testcapi
def f(): yield _testcapi.stack_pointer()
print(_testcapi.stack_pointer() - next(f()))

      Stack usage depending on the compiler and compiler optimization level:

      • clang -O0: 9,104 bytes
      • clang -Og: 736 bytes
      • gcc -O0: 6,784 bytes
      • gcc -Og: 624 bytes

      -O0 allocates around 10x more memory.

      Moreover, "./configure --with-pydebug CC=clang" uses -O0 in CFLAGS, because "clang --help" output doesn't containt "-Og". I'm working on a configure change to use -Og on clang which supports it.

    2. vstinner
      added
      3.11only security fixes
      buildThe build process and cross-build
      performancePerformance or resource usage
      on Feb 1, 2022

    3. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      #75235 enables -Og when using clang and ./configure --with-pydebug and so the example uses 736 bytes instead of 9,104 bytes.

    4. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      This issue is a follow-up of bpo-46542 "test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot".

    5. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      Previous issues about stack memory usage, work done in 2017:

      • bpo-28870: Reduce stack consumption of PyObject_CallFunctionObjArgs() and like
      • bpo-29227: Reduce C stack consumption in function calls
      • bpo-29465: Modify _PyObject_FastCall() to reduce stack consumption
        29464

      I summarized the results in the "Stack consumption" section of my article: https://vstinner.github.io/contrib-cpython-2017q1.html

    6. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      See also bpo-30866: "Add _testcapi.stack_pointer() to measure the C stack consumption".

    7. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      stack_overflow-4.py: Update script from bpo-30866 to measure stack memory usage before Python crash or raises a RecursionError.

      I had to modify the script since calling a Python function from a Python function no longer allocates (additional) memory on the stack! See bpo-45256 "Remove the usage of the C stack in Python to Python calls".

    8. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      stack_overflow-4.py output depending on the compiler and compiler flags.

      gcc -O3 (./configure):
      ---
      test_python_call: 11904 calls before crash, stack: 704.1 bytes/call
      test_python_iterator: 17460 calls before crash, stack: 480.0 bytes/call
      test_python_getitem: 245760 calls before recursion error, stack: 0.2 bytes/call

      => total: 275124 calls, 1184.3 bytes per call
      ---

      It's better than stack memory usage in 2017: https://bugs.python.org/issue30866#msg297826

      clang -O3 (./configure CC=clang):
      ---
      test_python_call: 10270 calls before crash, stack: 816.1 bytes/call
      test_python_iterator: 14155 calls before crash, stack: 592.0 bytes/call
      test_python_getitem: 245760 calls before recursion error, stack: 0.3 bytes/call

      => total: 270185 calls, 1408.4 bytes per call
      ---

      clang allocates a little bit more memory on the stack than gcc.

      I didn't try PGO or LTO yet.

    9. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      New changeset 0515eaf by Victor Stinner in branch 'main':
      bpo-46600: ./configure --with-pydebug uses -Og with clang (GH-31052)
      0515eaf

    10. pablogsal commented on Feb 1, 2022

      @pablogsal
      pablogsal
      on Feb 1, 2022
      Member

      PR 31052 seems to have broken a bunch of buildbots. If no fix is provided in 24 hours, we will need to revert :(

    11. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      PR 31052 seems to have broken a bunch of buildbots. If no fix is provided in 24 hours, we will need to revert :(

      test_gdb fails if Python is built with clang -Og. I don't think that it's a regression. It's just that previously, buildbots using clang only build Python with -O0 or -O3.

      I'm investigating the test_gdb issue: it's easy to reproduce on Linux (clang 13.0.0). I may skip test_gdb is Python is built with clang -Og.

    12. vstinner commented on Feb 1, 2022

      @vstinner
      vstinner
      on Feb 1, 2022
      MemberAuthor

      New changeset bebaa95 by Victor Stinner in branch 'main':
      bpo-46600: Fix test_gdb.test_pycfunction() for clang -Og (GH-31058)
      bebaa95

    13. methane commented on Feb 2, 2022

      @methane
      methane
      on Feb 2, 2022
      Member

      FWIW, it seems -O0 don't merge local variables in different path or lifetime.

      For example, see _Py_abspath

          if (path[0] == '\0' || !wcscmp(path, L".")) {
       wchar_t cwd[MAXPATHLEN + 1];
       //(snip)
    }
    //(snip)
    wchar_t cwd[MAXPATHLEN + 1];

      wchar_t is 4bytes and MAXPATHLEN is 4096 on Linux. So each cwd is 16388bytes.
      -Og allocates 32856 bytes for it and -Og allocates 16440 bytes for it.

      I don't know what is the specific optimization flag in -Og do merge local variable, but I think -Og is very important for _PyEval_EvalFrameDefault() since it has many local variables in huge switch-case statements.
      -Og allocates 312 bytes for it and -O0 allocates 8280 bytes for it.

      By the way, clang 13 has -fstack-usage option like gcc, but clang 12 don't have it.
      Since Ubuntu 20.04 have only clang 12, I use -fstack-size-segment and https://github.com/mvanotti/stack-sizes to get stack size.

    14. vstinner commented on Feb 2, 2022

      @vstinner
      vstinner
      on Feb 2, 2022
      MemberAuthor

      For example, see _Py_abspath

      For functions which are commonly called in Python at runtime, it may be worth it to manually merged large local variables to save a few bytes on the stack when Python is built with -O0. For _Py_abspath(), this function is only called at startup, if I recall correctly, so it should be a big issue in practice.

    15. methane commented on Feb 3, 2022

      @methane
      methane
      on Feb 3, 2022
      Member

      I didn't mean _Py_abspath is problem. I just used it to describe why -O0 and -Og is so different.

      We can reduce stack usage of it easily, but it is not a problem than _PyEval_EvalFrameDefault.
      It is difficult to reduce stack usage of _PyEval_EvalFrameDefault with -O0.

    16. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    17. iritkatriel commented on Sep 12, 2022

      @iritkatriel
      iritkatriel
      on Sep 12, 2022
      Member

      Is there anything left to do here?

    18. iritkatriel
      added
      pendingThe issue will be closed if no feedback is provided
      on Sep 12, 2022

    19. vstinner commented on Sep 13, 2022

      @vstinner
      vstinner
      on Sep 13, 2022
      MemberAuthor

      There is always room for enhancement :-) But for now, IMO merged changes are enough to make the issue less complicated.

      The main change is to use -Og when Python is built in debug mode by clang.

    20. vstinner
      closed this as completedon Sep 13, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.11only security fixesbuildThe build process and cross-buildpendingThe issue will be closed if no feedback is providedperformancePerformance or resource usage

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions