Repository navigation
Cannot build arm64 arch on macOS / M1 host #40302
Description
Activity
That's weird. Looking at the output, it seems that our configure script detected the host arch as
x64and enabled cross-compilation:g++ -o /Users/aron/iPlugDev/iPlug2/Dependencies/node/out/Release/obj.host/...@targos Sorry for the delay, I've been busy with my day job ;) Here goes the content of the
config.gypi. Also, your assumptions seem to be correct from what I can see: The script determinesx86as the host architecture which is def. not correct in reality. However, I now have an idea why this happens:My whole Mac has been migrated instead of freshly installed. This means that I've had
homebrewinstalled on my Intel Mac before and installed theautomake/autoconftoolchain before, running on the Intel machine; I then ran the migration tool and now I might be running a toolchain that has been cross-compiled by Rosetta2. I'm not sure about this however. It might also be the default case with a fresh install on M1 under other circumstances and not be such of an edge case. Well, def.lipo -info /usr/bin/makegives meArchitectures in the fat file: /usr/bin/make are: x86_64 arm64e.That said, depending on the implementation of
host_arch_cc()which is used inconfigure.pyto determine the host arch (maybe its just checking the underlaying binary format to check the host architecture in a sense of "I'm x86 so the host must be x86 too!"?), it must come to the wrong conclusion in my case. It might be safer in general when something likeuname -aor some system/toolchain call would be used on unixoid host OSs than trying to assume it from the binary format the compiler toolchain is running on which can be false-positive in cases like Rosetta2. Wdyt?I'll triage in a few hours again, try to proof my point, fix it on my side and come up with a PR to fix this. However, introducing changes in this line
host_arch = host_arch_win() if os.name == 'nt' else host_arch_cc()could become a potential landmine on other host that I cannot test. Any change here should undergo a carefully test procedure imho; idk if we have integration tests for this, but that would be amazing... :)Anyways, here is my local
config.gypi:
https://pastebin.com/9asbsHNPI think I found the bug.
def host_arch_cc()
It prints:
k[i] __x86_64__ i 1 rtn x64Which leads to:
host_arch: x64 target_arch: arm64This all boils down to:
k = cc_macros(os.environ.get('CC_host'))Update:
CC_hostisNone. This leads toCCbeing executed, which defaults to theccexecutable in path which is:Arons-MBP:node-16.10.0 aron$ which cc /usr/bin/cc
And which is itself dual
x86_64 arm64e:Arons-MBP:node-16.10.0 aron$ file /usr/bin/cc /usr/bin/cc: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e] /usr/bin/cc (for architecture x86_64): Mach-O 64-bit executable x86_64 /usr/bin/cc (for architecture arm64e): Mach-O 64-bit executable arm64e
And it probably selects the first architecture it finds from that one... investigating further
Guess I found it... the output of that parsed
cc -dM -E -call on M1 is['#define _LP64 1',...]The architecture
_LP64is not part of the mapping:matchup = { '__aarch64__' : 'arm64', '__arm__' : 'arm', '__i386__' : 'ia32', '__MIPSEL__' : 'mipsel', '__mips__' : 'mips', '__PPC64__' : 'ppc64', '__PPC__' : 'ppc64', '__x86_64__' : 'x64', '__s390x__' : 's390x', '__riscv' : 'riscv', }
Which ultimately leads to the wrong determination of the host architecture.
Adding:
'_LP64' : 'arm64',
`Will fix this nasty bug and lead to the correct result: host_arch = arm64
Would you accept a PR @targos ?
Best,
AronI'm not really a macOS user but @AshCripps set up our fat binary builds
. I believe this is all run on a M1 mac outside of Rosetta 2.Lines 981 to 1055 in 7f2cb44
Reacted by Aron Homberg and Sophie BremerI do most of my nodejs work with an M1 MBP and it works fine for me, so this is not a general issue
Reacted by Aron HombergEven if you do get the x64 build to run I believe it fails on a V8 compilation step which is why we do arm64 -> x64 and not the other way round in our release builds.
Reacted by Aron HombergI do most of my nodejs work with an M1 MBP and it works fine for me, so this is not a general issue
Yes, I came to the same conclusion now. This only happens in dual-arch toolchain cases where Rosetta 2 emulates an
x86_64platform environment.Summing up, with the new patch in PR, it works well in this edge case as well. Compilation results in:
Arons-MBP:node-16.10.0 aron$ lipo -info out/Release/libv8_init.a Non-fat file: out/Release/libv8_init.a is architecture: arm64 Arons-MBP:node-16.10.0 aron$ lipo -info out/Release/libnode.a Non-fat file: out/Release/libnode.a is architecture: arm64 Arons-MBP:node-16.10.0 aron$ lipo -info out/Release/libuv.a Non-fat file: out/Release/libuv.a is architecture: arm64 Arons-MBP:node-16.10.0 aron$
Even if you do get the x64 build to run I believe it fails on a V8 compilation step which is why we do arm64 -> x64 and not the other way round in our release builds.
@AshCripps When I was not specifying the
--dest-cpu=arm64(without my patch) it went down thex86_64road and built withtarget_arch = x86_64andhost_arch = x86_64which resulted in a (from the new understanding) surprising result of a validx86_64build. I can try to double-check and verify this if you're interested.Only specifying
--dest-cpu=arm64(without my patch) led totarget_arch = arm64whilehost_archwas still wrongly identified asx86_64. This resulted in an arch mismatch triggering the cross-compilation and settingx86_64forv8-- it then of course broke the compilation becausev8tried to compilex86_64inline assembly on an ARM cpu.The current behaviour (with my patch) is:
You don't have to specify any--dest-cpuor anything; I just run./configure && make-- it will detect thehost_archcorrectly, even in this edge-case situation, and give me non-fat libs and executables having onlyarm64as a result@AshCripps I'm actually running all permutations I can imagine now... just to re-verify;
a) without the patch, no--dest-cpuflag; but with the edge-case of Rosetta 2 on the toolchain... update: x86_64, non-fat
b) without the patch, with--dest-cpu=arm64(should lead to the error above): update: crash: tries to build inline assembly for x86_64 (v8) just as above
c) with the patch, no--dest-cpuflag; should lead to the expected result of a non-fatarm64build; update: arm64 non-fat
d) with the patch, and--dest-cpuflag set toarm64; should have no effect; equal to c); update: equal to c)
e) with the patch, and--dest-cpuflag set tox86_64; that will be interesting; update: crashes; tries to build openssl with inline asm for x86_64- added a commit that references this issue
on Oct 4, 2021 a) without the patch, no
--dest-cpuflag; but with the edge-case of Rosetta 2 on the toolchain...I'd kind of expect a Rosetta 2 toolchain to build x64 binaries -- isn't the point of Rosetta 2 to pretend to be x64?
@richardlau Well, that is a valid argument from my point of view; actually, I think both mental models could work; one could argue that Rosetta 2s intention is to make x86_64 work on arm64, therefore the output of a compilation should be arm64 by default. Or one could argue that Rosetta runs x86_64 on arm and therefore the output should be x86_64. Hmm...
But even if we take the second mental model, and assume that when running via Rosetta, the compilation output should be x86_64, then it would be nice if the compilation wouldn't crash when you specify the
--dest-cputargetarm64, right?To make this work, my patch would need another condition to check for
--dest-cpuas wellAlternatively, the PR and issue could be closed, because it's an edge case; but then I'd suggest to document it at least in the build readme for Mac / M1. I could also come up with a small PR for that.
Wdyt?
But even if we take the second mental model, and assume that when running via Rosetta, the compilation output should be x86_64, then it would be nice if the compilation wouldn't crash when you specify the
--dest-cputargetarm64, right?Yeah, for this I would expect the build to be cross-compiling for arm64 (but with a x64 host toolchain). If that doesn't work (I think @AshCripps may have alluded to it not working when we tried before) then there's possibly a bug somewhere in the build scripts but it's unlikely to be
configure. It may even be an issue upstream in V8.I think this is the wrong way to solve the issue - as far as I understand you are trying to compile arm64 from within an x64 environment (rosetta) right? To me the fix should involve the
--dest-cpuflag and not fudging the detection to detect the host underneath rosetta - cause that could lead to awkwardness when trying to build x64 inside rosetta. In fact this will break our release builds.Even if you do get the x64 build to run I believe it fails on a V8 compilation step which is why we do arm64 -> x64 and not the other way round in our release builds.
This is what I found as well, but it seems to be because V8 is being compiled with
CC_hostbut passes-DV8_TARGET_ARCH_ARM64which corresponds to the--dest-cpu=arm64flag. It seems like this should be fixable, but this is kind of a tangent so I'll open a separate issue -- #40350This issue has been marked as stale due to 210 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.This issue has been automatically closed after 30 days of inactivity following its stale status (no activity for a total of 120 days).
If this is still relevant, feel free to reopen it or leave a comment with additional details so we can continue the discussion.
Version
16.10.0
Platform
Darwin dev 20.6.0 Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:20 PDT 2021; root:xnu-7195.141.6~3/RELEASE_ARM64_T8101 arm64
Subsystem
No response
What steps will reproduce the bug?
How often does it reproduce? Is there a required condition?
Always; deterministic
What is the expected behavior?
Main binary and dependent libraries are working and output
arm64onlipo -infocallWhat do you see instead?
Additional information
v8 seems to be trying to build
src/heap/base/asm/x64instead of using inline assembly implementationsrc/heap/base/asm/arm64which is available.How to switch the target arch when compiling Node.js?
I'm embedding Node.js in a C++ application which needs to be as lightweight as possible. Therefore I'm trying to get a non-universal build containing only arm64 binary, also for the dependent libs (
libuv,v8etc).Am I overlooking something? I thought using
--dest-cpuwas quite the go-to flag...I'd also be happy about any comments helping me with this (hacks for the build scripts etc.)