Skip to content

Build failure with "--with-address-sanitizer" flag #121847

Description

@zack-alex

Bug report

Bug description:

When building Python with --with-address-sanitizer flag, the build fails. Leak sanitizer triggers when python is run.

Possible way to reproduce:

nix build --impure --expr \
  'with import <nixpkgs> {};
   python312.overrideAttrs (oldAttrs: {
     configureFlags = oldAttrs.configureFlags or [] ++ ["--with-address-sanitizer"];
   })'

Relevant portion of the build log:

...
./_bootstrap_python ./Programs/_freeze_module.py abc ./Lib/abc.py Python/frozen_modules/abc.h
./_bootstrap_python ./Programs/_freeze_module.py codecs ./Lib/codecs.py Python/frozen_modules/codecs.h
./_bootstrap_python ./Programs/_freeze_module.py io ./Lib/io.py Python/frozen_modules/io.h
./_bootstrap_python ./Programs/_freeze_module.py _collections_abc ./Lib/_collections_abc.py Python/frozen_modules/_collections_abc.h
./_bootstrap_python ./Programs/_freeze_module.py _sitebuiltins ./Lib/_sitebuiltins.py Python/frozen_modules/_sitebuiltins.h
./_bootstrap_python ./Programs/_freeze_module.py genericpath ./Lib/genericpath.py Python/frozen_modules/genericpath.h
./_bootstrap_python ./Programs/_freeze_module.py ntpath ./Lib/ntpath.py Python/frozen_modules/ntpath.h
./_bootstrap_python ./Programs/_freeze_module.py posixpath ./Lib/posixpath.py Python/frozen_modules/posixpath.h
./_bootstrap_python ./Programs/_freeze_module.py os ./Lib/os.py Python/frozen_modules/os.h
./_bootstrap_python ./Programs/_freeze_module.py site ./Lib/site.py Python/frozen_modules/site.h
./_bootstrap_python ./Programs/_freeze_module.py stat ./Lib/stat.py Python/frozen_modules/stat.h
./_bootstrap_python ./Programs/_freeze_module.py importlib.util ./Lib/importlib/util.py Python/frozen_modules/importlib.util.h
./_bootstrap_python ./Programs/_freeze_module.py importlib.machinery ./Lib/importlib/machinery.py Python/frozen_modules/importlib.machinery.h
./_bootstrap_python ./Programs/_freeze_module.py runpy ./Lib/runpy.py Python/frozen_modules/runpy.h
./_bootstrap_python ./Programs/_freeze_module.py __hello__ ./Lib/__hello__.py Python/frozen_modules/__hello__.h
./_bootstrap_python ./Programs/_freeze_module.py __phello__ ./Lib/__phello__/__init__.py Python/frozen_modules/__phello__.h
./_bootstrap_python ./Programs/_freeze_module.py __phello__.ham ./Lib/__phello__/ham/__init__.py Python/frozen_modules/__phello__.ham.h
./_bootstrap_python ./Programs/_freeze_module.py __phello__.ham.eggs ./Lib/__phello__/ham/eggs.py Python/frozen_modules/__phello__.ham.eggs.h
./_bootstrap_python ./Programs/_freeze_module.py __phello__.spam ./Lib/__phello__/spam.py Python/frozen_modules/__phello__.spam.h
./_bootstrap_python ./Programs/_freeze_module.py frozen_only ./Tools/freeze/flag.py Python/frozen_modules/frozen_only.h

=================================================================
[1m[31m==408115==ERROR: LeakSanitizer: detected memory leaks
[1m[0m
[1m[34mDirect leak of 15657 byte(s) in 318 object(s) allocated from:
[1m[0m    #0 0x7ffff79fd04f in __interceptor_malloc (/nix/store/c6r62m84hywf4i6qq1h28f13zv38yqyp-gcc-13.3.0-lib/lib/libasan.so.8+0xdc04f)
    #1 0x6efb6e in PyUnicode_New Objects/unicodeobject.c:1251
    #2 0x76ef10 in PyUnicode_New Objects/unicodeobject.c:1200
    #3 0x76ef10 in unicode_decode_utf8 Objects/unicodeobject.c:4693
    #4 0x4eb9ee in _PyPegen_decode_string Parser/string_parser.c:185
    #5 0x4eb9ee in _PyPegen_decode_string Parser/string_parser.c:182
    #6 0x4ec294 in _PyPegen_parse_string Parser/string_parser.c:273
    #7 0x44c06e in _PyPegen_constant_from_string Parser/action_helpers.c:1413
    #8 0x49e4df in string_rule Parser/parser.c:16242
    #9 0x49e4df in _tmp_259_rule Parser/parser.c:40700
    #10 0x49e4df in _loop1_115_rule Parser/parser.c:32163
    #11 0x49e4df in strings_rule Parser/parser.c:16294
    #12 0x45eecb in atom_rule Parser/parser.c:14690
    #13 0x464a8f in primary_raw Parser/parser.c:14328
    #14 0x464a8f in primary_rule Parser/parser.c:14126
    #15 0x4659f9 in await_primary_rule Parser/parser.c:14080
    #16 0x4668e7 in power_rule Parser/parser.c:13956
    #17 0x4668e7 in factor_rule Parser/parser.c:13906
    #18 0x467d8f in term_raw Parser/parser.c:13747
    #19 0x467d8f in term_rule Parser/parser.c:13509
    #20 0x4692c4 in sum_raw Parser/parser.c:13463
    #21 0x4692c4 in sum_rule Parser/parser.c:13342
    #22 0x46a224 in shift_expr_raw Parser/parser.c:13302
    #23 0x46a224 in shift_expr_rule Parser/parser.c:13181
    #24 0x46afa4 in bitwise_and_raw Parser/parser.c:13141
    #25 0x46afa4 in bitwise_and_rule Parser/parser.c:13059
    #26 0x46bbc4 in bitwise_xor_raw Parser/parser.c:13019
    #27 0x46bbc4 in bitwise_xor_rule Parser/parser.c:12937
    #28 0x46c7e4 in bitwise_or_raw Parser/parser.c:12897
    #29 0x46c7e4 in bitwise_or_rule Parser/parser.c:12815
    #30 0x46d4a0 in comparison_rule Parser/parser.c:12055
    #31 0x46d4a0 in inversion_rule Parser/parser.c:12006
    #32 0x4703be in conjunction_rule Parser/parser.c:11883
    #33 0x4713ae in disjunction_rule Parser/parser.c:11795
    #34 0x4729f4 in expression_rule Parser/parser.c:11083
    #35 0x476061 in kvpair_rule Parser/parser.c:16754
    #36 0x4ae2ce in double_starred_kvpair_rule Parser/parser.c:16714
    #37 0x4ae2ce in _loop0_118_rule Parser/parser.c:32287
    #38 0x4ae2ce in _gather_117_rule Parser/parser.c:32354
    #39 0x45fe55 in double_starred_kvpairs_rule Parser/parser.c:16642
    #40 0x45fe55 in dict_rule Parser/parser.c:16561
    #41 0x45fe55 in _tmp_96_rule Parser/parser.c:30838
    #42 0x45fe55 in atom_rule Parser/parser.c:14772
    #43 0x4b2e1b in t_primary_raw Parser/parser.c:18967
    #44 0x4b2e1b in t_primary_rule Parser/parser.c:18757
    #45 0x4b41d4 in target_with_star_atom_rule Parser/parser.c:18257
    #46 0x4b7059 in star_target_rule Parser/parser.c:18200
    #47 0x4b7769 in star_targets_rule Parser/parser.c:17942
    #48 0x4c8bba in _tmp_250_rule Parser/parser.c:40232
    #49 0x4c8bba in _loop1_14_rule Parser/parser.c:25810
    #50 0x4c8bba in assignment_rule Parser/parser.c:2360
    #51 0x4cc0f3 in simple_stmt_rule Parser/parser.c:1707
...

CPython versions tested on:

3.10, 3.12

Operating systems tested on:

Linux

Activity

  1. sobolevn commented on Jul 16, 2024

    @sobolevn
    Member

    Can you please reproduce the same without nix?

  2. zack-alex commented on Jul 16, 2024

    @zack-alex
    Author

    Nix is just one of thousands of linux distributions out there. What's the problem with it and what distribution would you like me to use?

  3. sobolevn commented on Jul 16, 2024

    @sobolevn
    Member

    Oh, sorry. I thought that nix is a build tool, alternative to our Makefile.

  4. zack-alex commented on Jul 16, 2024

    @zack-alex
    Author

    Yeah, it's a high-level package manager, it uses the build system of the upstream, configure + make in this case. Here is a log if it's of any help: log.txt

  5. encukou commented on Sep 10, 2025

    @encukou
    Member

    This particular leak looks cosmetic: _bootstrap_python is a build-only tool that doesn't run long enough for leaks to be a concern.

    If send a fix for the main branch I can merge it, but we're unlikely to backport it to 3.12.

  6. vstinner commented on Sep 10, 2025

    @vstinner
    Member

    I created #138756 for PyInitConfig_SetStr() memory leak.

  7. vstinner commented on Sep 10, 2025

    @vstinner
    Member

    CPython versions tested on: 3.10, 3.12

    Sadly these branches don't get bugfixes anymore: https://devguide.python.org/versions/

    I cannot reproduce this issue on 3.13 branch.

    On the 3.14 and main branches, I'm affected by #138756 bug, but make completes without error if I apply my #138759 fix.

    I close the issue. Thanks for your report anyway.

  8. vstinner commented on Sep 10, 2025

    @vstinner
    Member

    I cannot reproduce this issue on 3.13 branch.

    FYI I built Python with ./configure --with-pydebug --with-address-sanitizer && make.

  9. vstinner commented on Sep 10, 2025

    @vstinner
    Member

    Ah, without --with-pydebug, immortal interned strings are not released at Python exit. Extract of _PyUnicode_ClearInterned():

            case SSTATE_INTERNED_IMMORTAL:
                /* Make immortal interned strings mortal again.
                 *
                 * Currently, the runtime is not able to guarantee that it can exit
                 * without allocations that carry over to a future initialization
                 * of Python within the same process. i.e:
                 *   ./python -X showrefcount -c 'import itertools'
                 *   [237 refs, 237 blocks]
                 *
                 * This should remain disabled (`Py_DEBUG` only) until there is a
                 * strict guarantee that no memory will be left after
                 * `Py_Finalize`.
                 */
    

    See issue gh-113993 and change GH-120520.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildThe build process and cross-buildtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions