Repository navigation
Yarn's bundled node-gyp broken #346
Description
Activity
You can refer to: https://circleci.com/gh/coralproject/talk/1921 with details of why the build failed.
To me it failed only because the build was trying to override the included yarn. I don't see any node-gyp error.
As you can see, several other build because of trying to install yarn via npm: #337 (comment)
I linked the wrong build logs :( below is the relevant error:
Step 9/11 : RUN yarn install --production ---> Running in c936d65bcc45 yarn install v0.21.3 [1/4] Resolving packages... [2/4] Fetching packages... warning fsevents@1.0.17: The platform "linux" is incompatible with this module. info "fsevents@1.0.17" is an optional dependency and failed compatibility check. Excluding it from installation. [3/4] Linking dependencies... [4/4] Building fresh packages... error /usr/src/app/node_modules/bcrypt: Command failed. Exit code: 127 Command: sh Arguments: -c node-gyp rebuild Directory: /usr/src/app/node_modules/bcrypt Output: sh: 1: node-gyp: not found info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command. The command '/bin/sh -c yarn install --production' returned a non-zero code: 1Which was ran with the repo here https://github.com/coralproject/talk/tree/2c98aa20977155d09b08e927c574be1221d2d707 where I had removed my patch from the Dockerfile to make it the following:
FROM node:7.6 # Add node-gyp for bcrypt build support # RUN yarn global add node-gyp # Create app directory RUN mkdir -p /usr/src/app WORKDIR /usr/src/app # Setup the environment ENV NODE_ENV production ENV PATH /usr/src/app/bin:$PATH ENV TALK_PORT 5000 EXPOSE 5000 # Install app dependencies COPY package.json yarn.lock /usr/src/app/ RUN yarn install --production # Bundle app source COPY . /usr/src/app CMD [ "yarn", "start" ]Node-gyp is not included in node though so you would always need to install it no?
Previously, using
npm installit successfully installs.It doesn't install with yarn?
That's correct. With the following Dockerfile:
FROM node:7.6 # Add node-gyp for bcrypt build support # RUN yarn global add node-gyp # Create app directory RUN mkdir -p /usr/src/app WORKDIR /usr/src/app # Setup the environment ENV NODE_ENV production ENV PATH /usr/src/app/bin:$PATH ENV TALK_PORT 5000 EXPOSE 5000 # Install app dependencies COPY package.json yarn.lock /usr/src/app/ # RUN yarn install --production RUN npm install --production # Bundle app source COPY . /usr/src/app CMD [ "npm", "start" ]
I got no errors during install.
That's weird...I ran the same thing and it worked:
Sending build context to Docker daemon 2.048 kB Step 1/2 : FROM node:7.6 7.6: Pulling from library/node 693502eb7dfb: Pull complete 081cd4bfd521: Pull complete 5d2dc01312f3: Pull complete 54a5f7da9a4f: Pull complete f6ebc0704397: Pull complete cd487f72ac9f: Pull complete d3e88bf53e70: Pull complete 88dbf525e469: Pull complete Digest: sha256:34d21bab49105c1ab384e50432aa6ebd6dac69c2637378e7ecb6f40cda97118b Status: Downloaded newer image for node:7.6 ---> 7b74673e3e78 Step 2/2 : RUN yarn global add node-gyp ---> Running in 58721d12a3bd yarn global v0.21.3 [1/4] Resolving packages... [2/4] Fetching packages... [3/4] Linking dependencies... [4/4] Building fresh packages... success Installed "node-gyp@3.5.0" with binaries: - node-gyp Done in 5.15s. ---> d383b79b9c88 Removing intermediate container 58721d12a3bd Successfully built d383b79b9c88Is it possible that node-gyp installs but isn't working?
This is my issue:
When installing via yarn, I need to run:
yarn global add node-gyp yarn install --productionWhile with npm I can run:
npm install --productionI don't need to explicitly install
node-gyp, hence why it seems that the upstream issue with yarn (yarnpkg/yarn#2266) might be causing the issue now where when installed via yarn (specifically, the bundled version), thatnode-gypfails.Notice that in your example, you explicitly ran
yarn global add node-gypAs the second step.
This seems to be similar to your issue: yarnpkg/yarn#1848
I can confirm that after running:
yarn upgrade bcryptWhich resulted in the following changes:
diff --git a/package.json b/package.json index 740063dd..eaa15303 100644 --- a/package.json +++ b/package.json @@ -49,7 +49,7 @@ }, "homepage": "https://github.com/coralproject/talk#readme", "dependencies": { - "bcrypt": "^0.8.7", + "bcrypt": "^1.0.2", "body-parser": "^1.15.2", "cli-table": "^0.3.1", "commander": "^2.9.0", diff --git a/yarn.lock b/yarn.lock index 0331202e..f8a49205 100644 --- a/yarn.lock +++ b/yarn.lock @@ -1089,12 +1089,13 @@ bcrypt-pbkdf@^1.0.0: dependencies: tweetnacl "^0.14.3" -bcrypt@^0.8.7: - version "0.8.7" - resolved "https://registry.yarnpkg.com/bcrypt/-/bcrypt-0.8.7.tgz#bc3875a9afd0a7b2cd231a6a7f218a5ce156b093" +bcrypt@^1.0.2: + version "1.0.2" + resolved "https://registry.yarnpkg.com/bcrypt/-/bcrypt-1.0.2.tgz#d05fc5d223173e0e28ec381c0f00cc25ffaf2736" dependencies: bindings "1.2.1" - nan "2.3.5" + nan "2.5.0" + node-pre-gyp "0.6.32" big.js@^3.1.3: version "3.1.3" @@ -5006,7 +5007,11 @@ mute-stream@0.0.7: version "0.0.7" resolved "https://registry.yarnpkg.com/mute-stream/-/mute-stream-0.0.7.tgz#3075ce93bc21b8fab43e1bc4da7e8115ed1e7bab" -nan@2.3.5, nan@^2.3.0: +nan@2.5.0: + version "2.5.0" + resolved "https://registry.yarnpkg.com/nan/-/nan-2.5.0.tgz#aa8f1e34531d807e9e27755b234b4a6ec0c152a8" + +nan@^2.3.0: version "2.3.5" resolved "https://registry.yarnpkg.com/nan/-/nan-2.3.5.tgz#822a0dc266290ce4cd3a12282ca3e7e364668a08" @@ -5111,7 +5116,7 @@ node-libs-browser@^2.0.0: util "^0.10.3" vm-browserify "0.0.4" -node-pre-gyp@^0.6.29: +node-pre-gyp@0.6.32, node-pre-gyp@^0.6.29: version "0.6.32" resolved "https://registry.yarnpkg.com/node-pre-gyp/-/node-pre-gyp-0.6.32.tgz#fc452b376e7319b3d255f5f34853ef6fd8fe1fd5" dependencies:
It was successfully able to build! I'm still hesitant as the fact that the same package.json + yarn.lock file can build the project successfully without explicitly installing a package (node-gyp) yet yarn required some convincing to get it to run.
To me it still feels like there is an issue that yarn should sort out upstream to match the ability that node currently has to build these dependancies properly.
I'll close this for now, but feel free to ask any questions if anyone wants to explore this topic more.
I just tried switching to bcrypt to bcryptjs and it worked, there still is the warning about fsevents though.
It is indeed weird. I isolated the problem
This works:
FROM node:7.6 RUN npm install bcrypt@0.8.7
This doesn't:
FROM node:7.6 RUN yarn add bcrypt@0.8.7
Reacted by Franz Josef KaiserThat would be my conclusion as well. The fact that
0.8.7did not include"node-pre-gyp": "0.6.32"as a dep is why1.0.2worked. This issue is still persisting then.So when building a docker image descended from
node:6everything is really messed up:Step 6/7 : RUN yarn ---> Running in f1c656e5fc13 yarn install v0.21.3 [1/4] Resolving packages... [2/4] Fetching packages... [4/4] Building fresh packages... error /usr/src/app/node_modules/scrypt: Command failed. Exit code: 127 Command: sh Arguments: -c node-gyp rebuild Directory: /usr/src/app/node_modules/scrypt Output: sh: 1: node-gyp: not found info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command. ERROR: Service 'web' failed to build: The command '/bin/sh -c yarn' returned a non-zero code: 1I'd be curious if @Daniel15 has any ideas since he is a member of
yarnpkgand was involved with the PR that includedyarnwith this image. yarn/2266 seems to be the cause.All of our builds/containers have been broken for 2 days now. I'd love to see yarn get removed until
node:7, since this is basically a breaking change.Still having the issue. Currently using @wyattjoh fix.
Adding
RUN yarn global add node-gypbeforeyarn installworksThis is fixed in the next version of Yarn (currently only available as an RC, but there should be a stable release soon)
Reacted by Julio Canares and Franz Josef Kaiser@Daniel15 cool to know it 👍 I will try
I just wanted to note my experience here. I was using nvm on a debian box and getting what looked like this node-gyp issue. I had to uninstall nvm and use the system node to actually get everything working. But, I did not need to install node-gyp
The current version of yarn still has a pre-existing issue with the pre-bundled version of node-gyp related to yarnpkg/yarn#2266
A temporary fix relates to adding:
To our Dockerfile.