Repository navigation
--disable-gil & --enable-experimental-jit fails to build on M1 Mac #118844
Description
Activity
- changed the title
[-]--disable-gil & --wirh-experimental-jit fails to build on M1 Mac[/-][+]--disable-gil & --enable-experimental-jit fails to build on M1 Mac[/+]on May 9, 2024 Full compilation error:
Details
gcc -c -fno-strict-overflow -Wsign-compare -g -Og -Wall -D_Py_TIER2=1 -D_Py_JIT -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -DPy_BUILD_CORE -o Python/jit.o Python/jit.c In file included from Python/jit.c:392: ./jit_stencils.h:24576:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16a(code + 0x230, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:24577:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16b(code + 0x234, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:24578:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16c(code + 0x238, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:24579:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16d(code + 0x23c, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:24596:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow' patch_aarch64_16a(code + 0x2a8, (uintptr_t)&_PyCriticalSection_BeginSlow); ^ ./jit_stencils.h:24597:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow' patch_aarch64_16b(code + 0x2ac, (uintptr_t)&_PyCriticalSection_BeginSlow); ^ ./jit_stencils.h:24598:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow' patch_aarch64_16c(code + 0x2b0, (uintptr_t)&_PyCriticalSection_BeginSlow); ^ ./jit_stencils.h:24599:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow' patch_aarch64_16d(code + 0x2b4, (uintptr_t)&_PyCriticalSection_BeginSlow); ^ ./jit_stencils.h:45384:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16a(code + 0x1c0, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:45385:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16b(code + 0x1c4, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:45386:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16c(code + 0x1c8, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:45387:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16d(code + 0x1cc, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:45404:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow' patch_aarch64_16a(code + 0x238, (uintptr_t)&_PyCriticalSection_BeginSlow); ^ ./jit_stencils.h:45405:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow' patch_aarch64_16b(code + 0x23c, (uintptr_t)&_PyCriticalSection_BeginSlow); ^ ./jit_stencils.h:45406:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow' patch_aarch64_16c(code + 0x240, (uintptr_t)&_PyCriticalSection_BeginSlow); ^ ./jit_stencils.h:45407:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow' patch_aarch64_16d(code + 0x244, (uintptr_t)&_PyCriticalSection_BeginSlow); ^ ./jit_stencils.h:48060:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16a(code + 0x358, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:48061:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16b(code + 0x35c, (uintptr_t)&_PyCriticalSection_Resume); ^ ./jit_stencils.h:48062:49: error: use of undeclared identifier '_PyCriticalSection_Resume' patch_aarch64_16c(code + 0x360, (uintptr_t)&_PyCriticalSection_Resume); ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. make: *** [Python/jit.o] Error 1Reacted by Savannah OstrowskiThanks! I’m not sure if this is a build configuration that’s been given any thought yet...
or if it even makes sense right now (I believe our tier 2 optimization pipeline is disabled on free-threaded builds, but I’d need to check). Regardless, we’re headed towards a future with both of these features and should probably make sure that they at least build together.@savannahostrowski, interested in taking this too? I’m on my phone now, but I think we probably just need to
#includethe header where this is defined injit.c.Here' the code sample that is generated:
memcpy(data, data_body, sizeof(data_body)); patch_64(data + 0x0, instruction->oparg); memcpy(code, code_body, sizeof(code_body)); patch_aarch64_33rx(code + 0x30, (uintptr_t)data); patch_aarch64_26r(code + 0x16c, (uintptr_t)code + sizeof(code_body)); patch_aarch64_33rx(code + 0x190, (uintptr_t)data); patch_aarch64_26r(code + 0x1c8, instruction_starts[instruction->error_target]); patch_aarch64_16a(code + 0x1d0, (uintptr_t)&_PyEval_FormatExcUnbound); patch_aarch64_16b(code + 0x1d4, (uintptr_t)&_PyEval_FormatExcUnbound); patch_aarch64_16c(code + 0x1d8, (uintptr_t)&_PyEval_FormatExcUnbound); patch_aarch64_16d(code + 0x1dc, (uintptr_t)&_PyEval_FormatExcUnbound); patch_aarch64_16a(code + 0x1e8, (uintptr_t)&_PyCriticalSection_Resume); patch_aarch64_16b(code + 0x1ec, (uintptr_t)&_PyCriticalSection_Resume); patch_aarch64_16c(code + 0x1f0, (uintptr_t)&_PyCriticalSection_Resume); patch_aarch64_16d(code + 0x1f4, (uintptr_t)&_PyCriticalSection_Resume); patch_aarch64_16a(code + 0x200, (uintptr_t)&_PyThreadState_GetCurrent); patch_aarch64_16b(code + 0x204, (uintptr_t)&_PyThreadState_GetCurrent); patch_aarch64_16c(code + 0x208, (uintptr_t)&_PyThreadState_GetCurrent); patch_aarch64_16d(code + 0x20c, (uintptr_t)&_PyThreadState_GetCurrent);
Yep - feel free to assign to me @brandtbucher! I'll take a look tonight or tomorrow!
FWIW, I'm not sure this is M1-specific. It compiled just fine on my Dell laptop running XUbuntu 22.04. OTOH, on an old MacBook Air I have laying around (dual core Intel i5, MacOS 12.7.4), it failed to compile
Python/jit.cin the same fashion as on my M1 MBP. Maybe it's a MacOS issue?Reacted by Savannah Ostrowski- added a commit that references this issue
on May 11, 2024 @brandtbucher @savannahostrowski
I saw the close-out of the PR and apparent backport to 3.13. I can build fine on main, but compilation still fails for me on 3.13. On both main and 3.13 I used this configure command (grepped from config.status):
/bin/sh './configure' '--enable-experimental-jit=yes' '--disable-gil' '--with-openssl=/opt/homebrew' 'GDBM_CFLAGS=-I/opt/homebrew/opt/gdbm/include' 'GDBM_LIBS=-L/opt/homebrew/opt/gdbm/lib -lgdbm' 'LIBLZMA_CFLAGS=-I/opt/homebrew/include' 'LIBLZMA_LIBS=-L/opt/homebrew/lib -llzma' $ac_configure_extra_args --no-create --no-recursionI thought the PR was backported to 3.13, but maybe not? Diffing the main and 3.13 versions of
Python/jit.cI see this line is missing from the 3.13 version:#include "pycore_critical_section.h"Adding it allows the compilation to succeed. I don't see it in the
jit.csource on Github either.I saw the close-out of the PR and apparent backport to 3.13. I can build fine on main, but compilation still fails for me on 3.13.
Ah, sorry, looks like the backport failed! Re-running it now...
- added a commit that references this issue
on May 13, 2024 ...done!
Bug report
Bug description:
On my M1 Mac, if I configure like so:
./configure --disable-gil --enable-experimental-jitI get a compilation error:(followed by many more of the same error).
I don't recall if I've tried this configure combination before or not. I'm using Apple's clang on an up-to-date MacOS (14.4.1):
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
--disable-gil&--enable-experimental-jitfails to build #118935