Skip to content

Jit stencils on Windows contain debug data in 3.15 #142050

Closed
Closed
Jit stencils on Windows contain debug data in 3.15#142050
Labels
3.15pre-release feature fixes, bugs and security fixesOS-windowstopic-JITtype-bugAn unexpected behavior, bug, or error

Description

@chris-eibl
chris-eibl
opened on Nov 28, 2025

Bug report

Bug description:

E.g.

void
emit__NOP(
    unsigned char *code, unsigned char *data, _PyExecutorObject *executor,
    const _PyUOpInstruction *instruction, jit_state *state)
{
    // 
    // _NOP.o:     file format coff-x86-64
    // 0: '\x00\x00\x00\x00\x04\x00\x00\x00\xf1\x00\x00\x00<\x00\x00\x00\n\x00\x01\x11\x00\x00\x00\x00\x00\x00\x00\x00.\x00<\x11\x00\x00\x00\x00\xd0\x00\x15\x00\x01\x00\x04\x00\x00\x00\x16R\x00\x00\x00\x00\x00\x00clang version 21.1.4\x00\x00'
    // 4c: 00 00 00 00
    const unsigned char data_body[80] = {
        0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
        0xf1, 0x00, 0x00, 0x00, 0x3c, 0x00, 0x00, 0x00,
        0x0a, 0x00, 0x01, 0x11, 0x00, 0x00, 0x00, 0x00,
        0x00, 0x00, 0x00, 0x00, 0x2e, 0x00, 0x3c, 0x11,
        0x00, 0x00, 0x00, 0x00, 0xd0, 0x00, 0x15, 0x00,
        0x01, 0x00, 0x04, 0x00, 0x00, 0x00, 0x16, 0x52,
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x63, 0x6c,
        0x61, 0x6e, 0x67, 0x20, 0x76, 0x65, 0x72, 0x73,
        0x69, 0x6f, 0x6e, 0x20, 0x32, 0x31, 0x2e, 0x31,
        0x2e, 0x34,
    };
    memcpy(data, data_body, sizeof(data_body));
}

which is in fact empty on main for Linux (at least for me on WSL) or Windows on 3.14.

The above block is contained in every stencil :(

Clang now generates debug info into the object files. Don't know when it happened, maybe one of the clang updates. Doesn't matter anyway, the fix is easy, just ignore .debug$S COFF sections, which brings us back to

void
emit__NOP(
    unsigned char *code, unsigned char *data, _PyExecutorObject *executor,
    const _PyUOpInstruction *instruction, jit_state *state)
{
    // 
    // _NOP.o:     file format coff-x86-64
    // 0: '\x00\x00\x00\x00'
    // 4: 00 00 00 00
}

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Linked PRs

Activity

  1. chris-eibl
    added
    type-bugAn unexpected behavior, bug, or error
    OS-windows
    topic-JIT
    on Nov 28, 2025
  2. chris-eibl
    changed the title [-]Jit stencils on Windows contain debug data on Windows[/-] [+]Jit stencils on Windows contain debug data[/+] on Nov 28, 2025

  3. chris-eibl commented on Nov 30, 2025

    @chris-eibl
    chris-eibl
    on Nov 30, 2025
    MemberAuthor

    Don't know when it happened

    I've done some further investigation. Just changing back to

    _LLVM_VERSION = "20"
_EXTERNALS_LLVM_TAG = "llvm-20.1.8.0"

    in

    cpython/Tools/jit/_llvm.py

    Lines 14 to 15 in cd4d0ae

    _LLVM_VERSION = "21"
    _EXTERNALS_LLVM_TAG = "llvm-21.1.4.0"

    fixes this. So it must be related to the update of clang to 21.1.4.0 (#140973).

    Looking closer at the data

        // _NOP.o:     file format coff-x86-64
    // 0: '\x00\x00\x00\x00\x04\x00\x00\x00\xf1\x00\x00\x00<\x00\x00\x00\n\x00\x01\x11\x00\x00\x00\x00\x00\x00\x00\x00.\x00<\x11\x00\x00\x00\x00\xd0\x00\x15\x00\x01\x00\x04\x00\x00\x00\x16R\x00\x00\x00\x00\x00\x00clang version 21.1.4\x00\x00'
    // 4c: 00 00 00 00

    reveals clang version 21.1.4. Thus, I had high hopes that using -fno-ident might fix it. But unfortunately, this is not enough, still more metadata is written. So searching in the clang command line reference for metadata I found -fno-stack-size-section which doesn't change anything.

    Thus, I think just ignoring .debug$S COFF sections is the easiest way to go and I have verified using

    py Tools\jit\build.py -p PC -o jit-files aarch64-pc-windows-msvc x86_64-pc-windows-msvc i686-pc-windows-msvc

    that with this fix for all three configurations this metadata is no longer contained in the stencils (and was present in all of them before).

    I've checked again in WSL Ubuntu 24.04.3 LTS for clang 21.1.3 and on native Ubuntu 24.04.3 LTS for clang 21.1.7 that this does not happen for x86_64-pc-linux-gnu. I don't have a setup for cross builds, so unfortunately I cannot comment on aarch64-unknown-linux-gnu for Linux and I have no Mac.

    I have no idea why clang changed that only for Windows.

  4. chris-eibl commented on Nov 30, 2025

    @chris-eibl
    chris-eibl
    on Nov 30, 2025
    MemberAuthor

    As written in #142052 (comment), I am unsure about a backport, because

    • 3.14 uses clang 19.1.7 and is not affected. I've verified for aarch64-pc-windows-msvc x86_64-pc-windows-msvc i686-pc-windows-msvc.
    • no clang update is planned there anymore (I assume?)
    • the clang version used for the jit cannot (easily without touching _llmv.py) be changed. Make LLVM_VERSION configurable with env variable #138497 supports that only for configure based builds. And it is discouraged to use a different clang version, anyway. And the jit is experimental in 3.14 ...
    • but better safe than sorry? The fix is small and simple ...
  5. chris-eibl
    changed the title [-]Jit stencils on Windows contain debug data[/-] [+]Jit stencils on Windows contain debug data in 3.15[/+] on Nov 30, 2025
  6. chris-eibl
    added
    3.15pre-release feature fixes, bugs and security fixes
    on Nov 30, 2025

  7. chris-eibl commented on Dec 1, 2025

    @chris-eibl
    chris-eibl
    on Dec 1, 2025
    MemberAuthor

    I have no idea why clang changed that only for Windows.

    Ha, I found it, even though it is not part of the 21.1.0 release notes:
    llvm/llvm-project#142970: [CodeGen][COFF] Always emit CodeView compiler info on Windows targets

    MSVC always emits minimal CodeView metadata with compiler information, even when debug info is otherwise disabled. Other tools may rely on this metadata being present. For example, linkers use it to determine whether hotpatching is enabled for the object file.

    It is contained in clang since 21.1.0-rc1.
    Unfortunately, this is done unconditionally (at first it was planned to do it only when hotpatching is enabled) and there seems to be no switch to disable it.

  8. savannahostrowski
    added a commit that references this issue on Dec 3, 2025

    GH-142050: Jit stencils on Windows contain debug data (#142052)

    618dc36

  9. chris-eibl commented on Dec 4, 2025

    @chris-eibl
    chris-eibl
    on Dec 4, 2025
    MemberAuthor

    PR is merged, I think this can be closed.

  10. chris-eibl
    closed this as completedon Dec 4, 2025
  11. StanFromIreland
    added a commit that references this issue on Dec 6, 2025

    pythonGH-142050: Jit stencils on Windows contain debug data (python#1…

    0626f90
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.15pre-release feature fixes, bugs and security fixesOS-windowstopic-JITtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions