Please add binary for arm64

I'm trying to use this module on some arm64 devices and our build times are really long because there is no prebuilt binary for us.

node-pre-gyp ERR! Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v3.1.13/node-v57-linux-arm64.tar.gz 
node-pre-gyp ERR! Pre-built binaries not found for sqlite3@3.1.13 and node@8.9.0 (node-v57 ABI) (falling back to source compile with node-gyp)

I'm using the new LTS version of node.

You can reproduce this on a raspberry pi 3 using https://github.com/bamarni/pi64 which gives a 64-bit kernel. It does eventually compile from source, but every time uses about 15 minutes of time.

Hmm, actually, I'm having trouble build from source in some cases as well.

/srv/walnut/core/node_modules/sqlite3 # uname -m
aarch64
/srv/walnut/core/node_modules/sqlite3 # npm rebuild sqlite3
/srv/walnut/core/node_modules/sqlite3 # node -r sqlite3
module.js:538
    throw err;
    ^

Error: Cannot find module '/srv/walnut/core/node_modules/sqlite3/lib/binding/node-v57-linux-arm64/node_sqlite3.node'
    at Function.Module._resolveFilename (module.js:536:15)
    at Function.Module._load (module.js:466:25)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/srv/walnut/core/node_modules/sqlite3/lib/sqlite3.js:4:15)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)

I don't think it even tried to build from source when I initially ran npm install and npm rebuild doesn't seem to do anything.

Any ideas how I can debug node-prep-gyp issues here?

There is no ./lib/binding folder at all FWIW, which leads me to believe more it didn't try building the binary.

@creationix

Please add binary for arm64

Yes, this would really help users. To make providing an arm64 binary sustainable for each release, we'd need an automated way of doing this. Do you have suggestions on how to script building for arm64 on travis (or similar free CI infra)?

There is no ./lib/binding folder at all FWIW, which leads me to believe more it didn't try building the binary.

Odd, @creationix can you try passing --build-from-source to your npm rebuild command? Without that node-pre-gyp will be attempting to install from a remote binary each time, by default. Clearly something is wrong with the fallback though that I can't tell from your logs.

Assuming your CI server has support for qemu-aarch64-static and docker, something like the following should work:

FROM arm64v8/node:carbon
COPY qemu-aarch64-static /usr/bin/qemu-aarch64-static
RUN apt-get update
RUN apt-get install -y git build-essential
RUN git clone --depth=1 https://github.com/mapbox/node-sqlite3.git
WORKDIR /node-sqlite3
RUN make

See this article for more details on using qemu for cross-compiling with docker.

Another raspberry pi 3 node-sqlite user here. Pre-built would be wonderful 👍

One more raspberry pi 3 user here.

another user is added to the order

I guess this could be built on qemu , check this related change:
#1028

yet another non amd64 user

Relate to: #1028

and another ARM user here. I've been trying all sorts of tricks to get this to build on my raspi without success. Please add pre built binary to the repo... thanks a lot!

And another one user. Couldn't make it work on a raspberry pi zero w.

have you tried to build in docker ? may I also request this package to land in debian too

https://tracker.debian.org/pkg/node-sqlite3

Hi,
I can volunteer to create the build for the Pi. If someone knows the steps, please point me to it. I guess it would be:

where to get the original files
how to compile
where and how to push the compiled files.

Regards,

@songzhuang

Well I used this PR to build offline:

#1028

@songzhuang you can try to see how this can be adapted for arm64 for @k1r0s

Good luck

tools: Crossbuild for ARMv8-64

tools: Crossbuild for ARMv8-64 (#1147)

With docker buildx it is quite easy to build binaries for multiple platforms.

We are currently using it for building docker images for all platforms. We could incorporate it into the CI for easily generating pre-built binaries for SQLite.

I think it would be as easy as cloning this repository, running npm install --ignore-scripts and npx node-pre-gyp install --build-from-source and then upload the build artifacts.

A matrix could then cover all the platforms e.g. linux/arm/v6, linux/arm/v7, linux/arm64

Any update?

@louisabraham

@louisabraham I created this PR that adds binaries, feel free to review and test the binaries. #1362

Thank you so much, they work!

Any progress in making the binaries officially available?
I'm getting:

#13 11.19 > sqlite3@5.0.0 install /opt/node_app/node_modules/sqlite3
#13 11.19 > node-pre-gyp install --fallback-to-build
#13 11.19 
#13 11.35 node-pre-gyp WARN Using request for node-pre-gyp https download 
#13 11.81 node-pre-gyp WARN Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.0/napi-v3-linux-arm64.tar.gz 
#13 11.81 node-pre-gyp WARN Pre-built binaries not found for sqlite3@5.0.0 and node@14.15.1 (node-v83 ABI, glibc) (falling back to source compile with node-gyp)

Note linux/arm64 binaries are very important for the upcoming Docker Desktop for Apple Silicon M1 as those will run natively there.

@n1ru4l

@n1ru4l thanks for the PR. I could successfully generate the binaries using docker buildx, but I can't get npm and node-pre-gyp to use them.
When passing --source=napi-v3-linux-arm64.tar.gz, node-pre-gyp keeps trying to download it on s3 (and fails with error 403).

I'm almost there... How can I specify to node-pre-gyp that he should take my local file instead of fetching it over the distant repo?

You have to place it in a very specific folder so that the cache is hit. I don't remember the exact path though.

Thanks for the heads up. The location is shown in the error log file of npm just before the 403 response:

[...]
npm ERR! node-pre-gyp info check checked for "root/project/server/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64/node_sqlite3.node" (not found)

So all you have to do is to extract the node_sqlite3.node file from the tar.gz produced by docker buildx and place it in the node_module/sqlite3/lib/binding/napi-v3-linux-[ARCH]/node_sqlite3.node folder.
In my case I also had to delete first the project's node_modules folder to make it work (my previous commands had probably generated some invalid cache).

I hope they pull your PR soon to avoid this!

I wanted to workaround this while doing an npm ci which sadly meant I couldn't put the .node file in its final location.

This will let you host the binaries somewhere:

npm ci --node_sqlite3_binary_host_mirror=https://somehost

It'll try and download from http://somehost/sqlite3/v5.0.1/napi-v6-linux-arm64.tar.gz (the filename is the same name as the folder in lib/binding napi-v[NAPI_VERSION]-linux-[ARCH])
The tarball contains the folder from ../node_modules/sqlite3/lib/binding and in it the node_sqlite3.node file

@tom-boothman

Thanks for the workaround @tom-boothman, for some reason placing the file locally would sometimes not work for me, probably because I must use an older version of node-pre-gyp.
Having to upload the file on Internet while it's available on the very same device is quite an overkill but it does the job...

This has been shipped in master and will be released soon 🙂

Navigation Menu

Repository navigation

Please add binary for arm64 #903

3 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Please add binary for arm64 #903

Description

Activity

creationix commented on Nov 6, 2017

creationix commented on Nov 6, 2017

springmeyer commented on Nov 7, 2017

creationix commented on Nov 8, 2017

johnelliott commented on Nov 24, 2017

SridharRamasami commented on Mar 1, 2018

lucianodlf commented on Mar 17, 2018

rzr commented on Aug 8, 2018

k1r0s commented on Sep 16, 2018

rzr commented on Sep 17, 2018

Slos commented on Nov 8, 2018

dvilelaf commented on Nov 9, 2018

rzr commented on Nov 9, 2018

songzhuang commented on Nov 9, 2018

rzr commented on Nov 9, 2018

3 remaining items

n1ru4l commented on Jul 21, 2020

louisabraham commented on Aug 22, 2020

n1ru4l commented on Aug 22, 2020

louisabraham commented on Nov 1, 2020

gpetrov commented on Dec 13, 2020

dark-matter-matters commented on Mar 19, 2021

n1ru4l commented on Mar 19, 2021

dark-matter-matters commented on Mar 21, 2021

tom-boothman commented on May 28, 2021

dark-matter-matters commented on Jun 21, 2021

daniellockyer commented on Apr 14, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions