Repository navigation
Propagating build inputs #56
Description
My use case is being able to run bundle install in the shell for projects using Ruby gems with native dependencies that need to be compiled with gcc (say, libxml or Postgres client). This works when I specify those pacakges as buildInputs in pkgs.mkShell, but there's no option to do so with devshell and even when I hacked the source a bit to pass it down to buildEnv it doesn't help either – the culprit seems to be the list of environment variables passed down to the env (it's lacking the compiler-related envvars).
I'm not sure what's the best direction to take here (I'm a bit of a newb when it's coming to using nix as something other than a glorified ansible for my home infrastructure) but I think it's an useful use-case to cover.
Activity
90-008 commentedon Jan 16, 2021
I'm also having this issue. For example, putting
glibinbuildInputsinmkShellmakes glib'spkg-configfiles available, but withmkDevShellspackagesthis doesn't work. Or puttingmpvinpackagesthen trying to link tolibmpvalso won't work.zimbatm commentedon Jan 21, 2021
I had to remove the pkgs.stdenv machinery in order to re-build a clean user experience. This means that all of the standard nix-shell hooks are missing.
For now, the best option is to either use
pkgs.mkShell, or to extend the devshell with the proper environment variables. I'm working on adding those back as proper options.blaggacao commentedon Jan 22, 2021
I also wondered if it is generally a good idea to put (proper) project build dependencies into the devshell. An alternative place that would come to my mind would be a proper build / release nix infrastructure (outside of devshell) and then a little wrapper which would provide a commamd to the devshell (why not wrap
nix buildfor that purpose?) And also a package that exposes the environment.Not sure at all of this separation of concern is warranted, for sure it might be impractical of a sort. On the other hand one would probably also want those things be working also on stock CI (aka
nix build/nix check).I'll remain very interested to hear back from your thoughts on this.
90-008 commentedon Jan 22, 2021
I usually seperate build dependencies, build environment variables etc. stuff into a
common.nixfile, and import that file in my build and shell derivations. For now I think this works good enough. However I would like to be able to just use my build derivation's environment to also develop, which would remove the need for acommon.nix.zimbatm commentedon Jan 22, 2021
I have two sides to this:
Ideally, the devshell environment shouldn't be too fat. The issue is that each additional package makes it more expensive to load a fresh environment as Nix has to load a bunch of dependencies from the cache. Monorepos can quickly blow to multiple GB of data to download. This is a usability issue. Once everything is loaded into the /nix/store things go faster, but updating nixpkgs is not as rare as one might think. This also happens when switching development branches or bisecting old builds.
In order to solve this, I am looking to build an incremental Nix builder (yet another project). Other tools like Bazel might also be used. The advantage is that the initial pull becomes smaller, and then only fetch the subset of the tree you are interested in.
But we have to be pragmatic. In practice, LSP servers tend to need to have access to their respective compiler/interpreter. And the third-party libraries. Editors also want to be able to find the compiler to do their own checks. The story is not great there.
zimbatm commentedon Jan 22, 2021
I forgot to say that there is also #6 which was an experiment to propagate all the build inputs of a derivation to packages. The solution is not complete as it doesn't set LD_INCLUDE_PATH and friends.
90-008 commentedon Jan 24, 2021
This sounds pretty interesting, since you could "just" have a dev env for that package by propagating the build inputs. It would also solve "using build derivations env". This also makes me think of auto-generated dev shells, which would be ideal, although it may not be possible for every build process.
zimbatm commentedon Jan 24, 2021
That was one of the motivations for the CLI. At some point, I want to add some heuristics to
devshell initso it can detect the project structure and infer some good defaults based on that.Most current users are probably familiar with Nix already and know how to build derivations. The other types of users I want to attract are only going to use devshell to bring in the development dependencies, and then use the language native tools to build their project.
bbigras commentedon Feb 27, 2021
Any progress or workaround for this? I would like to use devshell with Rust projects needing some libs with
buildInputs.zimbatm commentedon Feb 27, 2021
@bbigras I added the
language.cmodule to handle C dependencies.With a bit of nix code is should be possible to inject those on both sides.
bbigras commentedon May 31, 2021
Do you mean adding the same code into
rust.nix?Builditluc commentedon Apr 28, 2022
Can someone provide an example of how to use
buildInputswith rust?