Repository navigation
node/openssl/ssl.h should compile without any additional defines #40575
Description
Activity
@danbev thoughts on this?
I haven't looked at node-gyp at all during the OpenSSL work so if this is something that is not covered by the OpenSSL tests in
test/addons (openssl-binding, openssl-client-cert-engine, or openssl-key-engine)then I've missed it I'm afraid.I added
OPENSSL_API_COMPATtonode.gypito avoid warnings due to the usage of deprecated APIs. We could remove this and live with the warnings but since we need to keep backward compatibility with OpenSSL 1.1.1 for some time and it might become annoying seeing these.@danbev You should be able to include
node/openssl/ssl.hfrom a standalone application from outside the Node.js treeThis is the test:
#include "openssl/ssl.h" int main() { return 0; }
If you have ever compiled a native module with 17.0.1 you will have this directory:
gcc -I${HOME}/.cache/node-gyp/17.0.1/include/node -o openssl openssl.c${HOME}/Cacheson macOSNormally, the OpenSSL of
node-gypshould be a drop-in replacement for a system-installed OpenSSL - it allows to build a native addon that uses OpenSSL without linking in a second copy in the Node executable - which might not work at allOn windows, and alpine linux docker image, updating to node 17...
After getting around
OPENSSL_API_COMPATby just defining it in CMakeList for it (and the binding.gyp), I ran intoerror "The requested API level higher than the configured API compatibility level"which OPENSSL_CONFIGURED_API is undefined... so being 0, anything I set as a compatibility level fails.
adding this to the top of ....cmake-js\node-x64\v17.0.1\include\node\openssl\macros.h
#include "./configuration.h"(which I don't think is a good or correct solution)
But even then I had to define the
OPENSSL_MIN_API=30000in my build configurationEdit: Sorry for being terse and vague.
After tracking down this, and getting some hints, I made forward progress... there's still missing symbols (
RSA_new()andRSA_free()for examples). There's a openssl/configuration.h file which is new in 17 that was not in 16. This configuration.h is not included by anything in openssl directory, and seems the user is supposed to include this file now too?Just chiming in that I'm also seeing this with node v17.x ("OPENSSL_API_COMPAT expresses an impossible API compatibility level").
Any (official) solutions to this? /cc @nodejs/crypto
Just chiming in that I'm also seeing this with node v17.x ("OPENSSL_API_COMPAT expresses an impossible API compatibility level").
Any (official) solutions to this? /cc @nodejs/crypto
"OPENSSL_API_COMPAT=10101"
and
#if NODE_MAJOR_VERSION >= 17 # include <openssl/configuration.h> #endif
before including anything from OpenSSL. If configuration.h is not defined than the configured version is undefined and becomes a '0' which everything is bigger than the configured level then.
@danbev How about including
#define OPENSSL_API_COMPAT=10101innode.h?Sorry about the late reply. I'm currently on a leave of absence and will be back working in April, and would be happy to take a closer look at this then (unless someone else beats me to it).
"OPENSSL_API_COMPAT=10101"
and
#if NODE_MAJOR_VERSION >= 17 # include <openssl/configuration.h> #endif
before including anything from OpenSSL. If configuration.h is not defined than the configured version is undefined and becomes a '0' which everything is bigger than the configured level then.
Unfortunately that doesn't seem to work for me, I still get the same error.
Looks like part of the problem in my particular case is that homebrew on macos is using OpenSSL 1.1.1 with their node v17.x formula, which causes problems for addons because of the OpenSSL 3.0 headers that get pulled in by node-gyp. The addon thinks it's compiling for OpenSSL 3.0 but then during runtime you can get various errors from dyld depending on if you are using new OpenSSL APIs or OpenSSL APIs that are now aliases in 3.0.
I've taken a look at this and the following is what I've found.
Looking at the the error when running
node-gyp build(with the addition of some
debugging of the macros involved to help troubleshoot this):/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:83:9: note: ‘#pragma message: OPENSSL_API_COMPAT: 30000’ 83 | #pragma message "OPENSSL_API_COMPAT: " OPENSSL_MSTR(OPENSSL_API_COMPAT) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:141:9: note: ‘#pragma message: OPENSSL_VERSION_MAJOR: 3’ 141 | #pragma message "OPENSSL_VERSION_MAJOR: " OPENSSL_MSTR(OPENSSL_VERSION_MAJOR) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:142:9: note: ‘#pragma message: OPENSSL_VERSION_MINOR: 0’ 142 | #pragma message "OPENSSL_VERSION_MINOR: " OPENSSL_MSTR(OPENSSL_VERSION_MINOR) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:143:9: note: ‘#pragma message: OPENSSL_VERSION: (3 * 1000 + OPENSSL_MINOR * 100)’ 143 | #pragma message "OPENSSL_VERSION: " OPENSSL_MSTR((OPENSSL_VERSION_MAJOR * 1000 + OPENSSL_MINOR * 100)) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:144:9: note: ‘#pragma message: OPENSSL_API_LEVEL: (30000)’ 144 | #pragma message "OPENSSL_API_LEVEL: " OPENSSL_MSTR(OPENSSL_API_LEVEL) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:145:9: note: ‘#pragma message: OPENSSL_API_COMPAT: 30000’ 145 | #pragma message "OPENSSL_API_COMPAT: " OPENSSL_MSTR(OPENSSL_API_COMPAT) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:146:9: note: ‘#pragma message: OPENSSL_CONFIGURED_API: OPENSSL_CONFIGURED_API’ 146 | #pragma message "OPENSSL_CONFIGURED_API: " OPENSSL_MSTR(OPENSSL_CONFIGURED_API) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:148:4: error: #error "The requested API level higher than the configured API compatibility level" 148 | # error "The requested API level higher than the configured API compatibility level" | ^~~~~ make: *** [example.target.mk:113: Release/obj.target/example/example.o] Error 1
Notice that
OPENSSL_CONFIGURED_APIhas not been set which is the cause of
the error message above.Now, macros.c has the following include:
#include <openssl/opensslconf.h> #include <openssl/opensslv.h>
In Node.js this is a "redirect" openssl header which choose different headers
depending of whether ASM of NO_ASM was configured:#if defined(OPENSSL_NO_ASM) # include "./opensslconf_no-asm.h" #else # include "./opensslconf_asm.h" #endif
And if we assume this using ASM this would end up in:
#elif defined(OPENSSL_LINUX) && defined(__x86_64__) # include "./archs/linux-x86_64/asm/include/openssl/opensslconf.h
And this file does not include configuration.h.
In OpenSSL the following headers are generated:
$ find . -name '*.in' ./include/openssl/conf.h.in ./include/openssl/lhash.h.in ./include/openssl/crmf.h.in ./include/openssl/err.h.in ./include/openssl/safestack.h.in ./include/openssl/ess.h.in ./include/openssl/opensslv.h.in ./include/openssl/ocsp.h.in ./include/openssl/pkcs12.h.in ./include/openssl/asn1.h.in ./include/openssl/bio.h.in ./include/openssl/x509.h.in ./include/openssl/x509v3.h.in ./include/openssl/cmp.h.in ./include/openssl/pkcs7.h.in ./include/openssl/srp.h.in ./include/openssl/x509_vfy.h.in ./include/openssl/asn1t.h.in ./include/openssl/ct.h.in ./include/openssl/fipskey.h.in ./include/openssl/ui.h.in ./include/openssl/configuration.h.in ./include/openssl/crypto.h.in ./include/openssl/cms.h.in ./include/openssl/ssl.h.in ./include/crypto/bn_conf.h.in ./include/crypto/dso_conf.h.in
And notice that
opensslconf.his not among these.
What I think is happening is that we have an old opensslconf.h which may
have been generated with past versions but is not generated any more, instead
configuration.his the file being generated.If we replace this header,
/.cache/node-gyp/17.8.0/include/node/openssl/opensslconf.h, with the header
from OpenSSL 3.0:$ cp opensslconf.h /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/opensslconf.hAnd the run the build again:
$ node-gyp build gyp info it worked if it ends with ok gyp info using node-gyp@9.0.0 gyp info using node@17.8.0 | linux | x64 gyp info spawn make gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ] make: Entering directory '/home/danielbevenius/work/nodejs/node-gyp-openssl-issue/build' CC(target) Release/obj.target/example/example.o In file included from /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/././archs/linux-x86_64/asm/include/openssl/ssl.h:21, from /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/./ssl_asm.h:11, from /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/ssl.h:4, from ../example.c:1: /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:83:9: note: ‘#pragma message: OPENSSL_API_COMPAT: 30000’ 83 | #pragma message "OPENSSL_API_COMPAT: " OPENSSL_MSTR(OPENSSL_API_COMPAT) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:141:9: note: ‘#pragma message: OPENSSL_VERSION_MAJOR: 3’ 141 | #pragma message "OPENSSL_VERSION_MAJOR: " OPENSSL_MSTR(OPENSSL_VERSION_MAJOR) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:142:9: note: ‘#pragma message: OPENSSL_VERSION_MINOR: 0’ 142 | #pragma message "OPENSSL_VERSION_MINOR: " OPENSSL_MSTR(OPENSSL_VERSION_MINOR) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:143:9: note: ‘#pragma message: OPENSSL_API_LEVEL: (30000)’ 143 | #pragma message "OPENSSL_API_LEVEL: " OPENSSL_MSTR(OPENSSL_API_LEVEL) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:144:9: note: ‘#pragma message: OPENSSL_API_COMPAT: 30000’ 144 | #pragma message "OPENSSL_API_COMPAT: " OPENSSL_MSTR(OPENSSL_API_COMPAT) | ^~~~~~~ /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:145:9: note: ‘#pragma message: OPENSSL_CONFIGURED_API: 30000’ 145 | #pragma message "OPENSSL_CONFIGURED_API: " OPENSSL_MSTR(OPENSSL_CONFIGURED_API) | ^~~~~~~ SOLINK_MODULE(target) Release/obj.target/example.node COPY Release/example.node make: Leaving directory '/home/danielbevenius/work/nodejs/node-gyp-openssl-issue/build' gyp info ok
Solution
So we should remove this template from Node.js:
$ git st . On branch opensslconf Changes to be committed: (use "git restore --staged <file>..." to unstage) deleted: deps/openssl/config/opensslconf.h deleted: deps/openssl/config/opensslconf.h.tmpl deleted: deps/openssl/config/opensslconf_no-asm.h
We can create the tar-headers file (by making some updates to the Makefile
to allow this to be built for a non-release):$ make node-v17.9.1.tar-headersFirst we can revert the headers:
$ node-gyp remove v17.8.0And running node-gyp rebuild should fail with the above error message.
And then unzip the new headers to replace the headers used by node-gyp:
$ tar xvzf node-v17.9.1-headers.tar.gz -C ~/.cache/node-gyp/17.8.0 --strip 1And this running node-gyp should succeed:
$ node-gyp build gyp info it worked if it ends with ok gyp info using node-gyp@9.0.0 gyp info using node@17.8.0 | linux | x64 gyp info spawn make gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ] make: Entering directory '/home/danielbevenius/work/nodejs/learning-nodejs/node-gyp-openssl-issue/build' make: Nothing to be done for 'all'. make: Leaving directory '/home/danielbevenius/work/nodejs/learning-nodejs/node-gyp-openssl-issue/build'
There are addons in Node.js that include openssl/ssl.h but this error does not
happen for those and I've not had time to find out why this is the case.- added 2 commits that reference this issue
on May 10, 2022 - added a commit that references this issue
on May 16, 2022
Version
17.0.1
Platform
5.11.0-38-generic #42~20.04.1-Ubuntu SMP
Subsystem
No response
What steps will reproduce the bug?
node/openssl/ssl.his exported in thenode-gypbundle to be used by native modules that need to link against the openssl symbols in NodeCurrently, when building Node.js itself some of these defines come from
openssl.gyp:OPENSSL_API_COMPATandOPENSSL_CONFIGURED_APIThis means that a native addon must also define them and must do so in a manner compatible with Node.js across the eventual future versions
How often does it reproduce? Is there a required condition?
No response
What is the expected behavior?
These defines should be present somewhere in an additional include file to be added to the
node-gypbundle distributionWhat do you see instead?
No response
Additional information
No response