Repository navigation
Switch to using xz compressed packages? #77
Description
Activity
Did a little experimenting with the recent 5.2.0 image. The build time on my mac is pretty close:
xz:
time docker build 5.2/ Sending build context to Docker daemon 8.192 kB Step 1 : FROM buildpack-deps:jessie ---> 8c1a2b85e52b Step 2 : RUN set -ex && for key in 9554F04D7259F04124DE6B476D5A82AC7E37093B 94AE36675C464D64BAFA68DD7434390BDBE9B9C5 0034A06D9D9B0064CE8ADF6BF1747F4AD2306D93 FD3A5288F042B6850C66B31F09FE44734EB7990E 71DCFD284A79C3B38668286BC97EC7A07EDE3FC1 DD8F2338BAE7501E3DD5AC78C273792F7D83545D ; do gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; done ---> Using cache ---> a04526abc207 Step 3 : ENV NPM_CONFIG_LOGLEVEL info ---> Using cache ---> 445ce4e025fe Step 4 : ENV NODE_VERSION 5.2.0 ---> Running in 4843fc11ed98 ---> dcb8cbcf2176 Removing intermediate container 4843fc11ed98 Step 5 : RUN curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.xz" && curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc" && gpg --verify SHASUMS256.txt.asc && grep " node-v$NODE_VERSION-linux-x64.tar.xz\$" SHASUMS256.txt.asc | sha256sum -c - && tar -xJf "node-v$NODE_VERSION-linux-x64.tar.xz" -C /usr/local --strip-components=1 && rm "node-v$NODE_VERSION-linux-x64.tar.xz" SHASUMS256.txt.asc ---> Running in ac226d3a3c3a % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 7965k 100 7965k 0 0 7729k 0 0:00:01 0:00:01 --:--:-- 7741k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3000 100 3000 0 0 22438 0 --:--:-- --:--:-- --:--:-- 22556 gpg: Signature made Wed Dec 9 05:23:57 2015 UTC using RSA key ID 7D83545D gpg: Good signature from "Rod Vagg <rod@vagg.org>" gpg: aka "Rod Vagg <r@va.gg>" gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: DD8F 2338 BAE7 501E 3DD5 AC78 C273 792F 7D83 545D node-v5.2.0-linux-x64.tar.xz: OK ---> eaf745af8e96 Removing intermediate container ac226d3a3c3a Step 6 : CMD node ---> Running in 534479b37d3a ---> cf8b379aafc6 Removing intermediate container 534479b37d3a Successfully built cf8b379aafc6 real 0m5.184s user 0m0.174s sys 0m0.015sgz:
time docker build 5.2/ Sending build context to Docker daemon 8.192 kB Step 1 : FROM buildpack-deps:jessie ---> 8c1a2b85e52b Step 2 : RUN set -ex && for key in 9554F04D7259F04124DE6B476D5A82AC7E37093B 94AE36675C464D64BAFA68DD7434390BDBE9B9C5 0034A06D9D9B0064CE8ADF6BF1747F4AD2306D93 FD3A5288F042B6850C66B31F09FE44734EB7990E 71DCFD284A79C3B38668286BC97EC7A07EDE3FC1 DD8F2338BAE7501E3DD5AC78C273792F7D83545D ; do gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; done ---> Using cache ---> a04526abc207 Step 3 : ENV NPM_CONFIG_LOGLEVEL info ---> Using cache ---> 445ce4e025fe Step 4 : ENV NODE_VERSION 5.2.0 ---> Using cache ---> dcb8cbcf2176 Step 5 : RUN curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.gz" && curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc" && gpg --verify SHASUMS256.txt.asc && grep " node-v$NODE_VERSION-linux-x64.tar.gz\$" SHASUMS256.txt.asc | sha256sum -c - && tar -xzf "node-v$NODE_VERSION-linux-x64.tar.gz" -C /usr/local --strip-components=1 && rm "node-v$NODE_VERSION-linux-x64.tar.gz" SHASUMS256.txt.asc ---> Running in 7679661ec442 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 11.3M 100 11.3M 0 0 4598k 0 0:00:02 0:00:02 --:--:-- 4598k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3000 100 3000 0 0 16549 0 --:--:-- --:--:-- --:--:-- 16483 gpg: Signature made Wed Dec 9 05:23:57 2015 UTC using RSA key ID 7D83545D gpg: Good signature from "Rod Vagg <rod@vagg.org>" gpg: aka "Rod Vagg <r@va.gg>" gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: DD8F 2338 BAE7 501E 3DD5 AC78 C273 792F 7D83 545D node-v5.2.0-linux-x64.tar.gz: OK ---> 51b37064e876 Removing intermediate container 7679661ec442 Step 6 : CMD node ---> Running in 4641de90ba67 ---> 4c6176e5f6d6 Removing intermediate container 4641de90ba67 Successfully built 4c6176e5f6d6 real 0m5.845s user 0m0.170s sys 0m0.013sOne thing to note for the above: I already had the base image layers on my mac. The build times on a "clean" docker host would be longer for both.
Image sizes!
The 5.2.0 image using the current gz compression:
$ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE node 5.2.0 53db8d9b8265 About a minute ago 642.6 MB buildpack-deps jessie a06b823c6a02 12 days ago 606.3 MBxz compression:
docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE node 5.2.0 a5758a2f0fd8 35 seconds ago 642.6 MB buildpack-deps jessie a06b823c6a02 12 days ago 606.3 MBHa! This makes sense though: in both cases it's a single RUN command that gets saved as a new layer. And the final size of node should be the same.
So there could be an improvement to the upstream build time of the image (and the different variants), but it might not be very significant. Probably worth running a few more tests.
Thanks for this thorough comparison @chorrell! As you mention this will not have an impact for most users (unless they build the image themselves).
Since
xzoffers a smaller download and comparable decompression time I am positive to usexzinstead ofgzip. The only drawback withxzis a higher memory constraint during decompression [1].[1] Quick Benchmark: Gzip vs Bzip2 vs LZMA vs XZ vs LZ4 vs LZO
[2] Featured on HN: For the love of bits, stop using gzip!
Thanks!
Yeah, that's pretty much how I lean too. I'll follow up with the upstream Docker people and see what they think.
A few things to note:
- There are no xz tarballs available for Node.js 0.10.x or 0.12.x
- The slim variant of 5.2.x and 4.2.x (which is based off buildpack-deps:jessie-curl) doesn't come with xz so it would need to be installed via the 'xz-utils' package.
- I'm definitely +1 to xz! 😄 FWIW, you can add "xz-utils" to $buildDeps for the slim variants ( https://github.com/nodejs/docker-node/blob/062488d2751e15c34996a2d76112975f39d98ae0/0.10/slim/Dockerfile#L19) and then it'll be removed afterwards too.
OK cool.
Yeah, unfortunately there are no xz compressed packages for 0.10 and 0.12. We'll need to do something similar for the 4.2.x and 5.3.x images
Next patch release of Node.js v0.10 and v0.12 will have
xzcompressed packages 💯Great!
I noticed that there are xz packages available. For instances of v4.2.3 (via https://nodejs.org/dist/v4.2.3/)
We've been using the .gz package in all out Dockerfiles. Should we switch to xz? The download would be faster, but I'm not sure what affect it would have on the image build. My understanding of xz is that compression takes a while, but decompression is fast. I'll do a bit of testing this week and see how things play out.