What is the best dev workflow around nix-shell?


#1

Hi! I’ve been a happy user of NixOS for quite some time, but I couldn’t quite polish my nix-shell workflow and I am wondering if you have any practical tips?

I think that my problems with nix-shell boil down to the fact that it serves two purposes:

  • it can be used to hack on derivations for nixos itself. I.e, if I want to hack on a nix package, I can use nix-shell to recreate the derivation environment and poke around.

  • it can be used to hack on the usual software, which is not package as a derivation. I.e, the foo project can have something like sudo apt-get libbar libbaz libqux in it’s README.md, and I can use nix-shell to install the said packages to make the usual make workflow work.

Now, I use nix-shell solely for the second use-case; I hack on projects which needs stuff like openssl, zlib and pkg-config to build. However, it seems to me that nix-shell was designed with the first use-case in mind. So, here’s the list of specific problems I face when using nix-shell:

  • it’s not really intuitive to me how to write default.nix. For Cargo, I have

    with import <nixpkgs> {}; {
      cargoEnv = stdenv.mkDerivation {
        name = "cargo";
        buildInputs = [ pkgconfig openssl cmake zlib libgit2 ];
        shellHook = ''
          export CFG_DISABLE_CROSS_TESTS=1
        '';
      }; 
    }
    

    It seems that name = "cargo" and cargoEnv bits don’t actually have any relevance, that is, it’s possible to use arbitrary names in their place. I generally just copy-paste this to over projects I am working on, because I can’t really write it from scratch.

  • today’s problem: when I run nix-shell --run zsh, my $SHELL changes to bash from zsh, which breaks other tools

  • problem a friend of mine faced: if you run nix-shell --run zsh, the gcc there is actually a wrapper which injects some nix-specific flags (for example, for hardening), and those flags might break compilation

  • there’s no way do create a “default, system-wide dev environment”, I need to launch nix-shell explicitly. This is not ideal when you want to create a second terminal tab, or launch an IDE from the launcher and not from the console.

So, does anyone know a better workflows for developing on NixOS? It seems like I am looking for something like "nix-shell for humans"…


#2

direnv integrates the nix-shell quite well and allows to keep the existing shell (for example zsh):

Personally I use the following shell function to make it easier to bootstrap:

I have some environments for various projects in my dotfiles as well:

direnv launches automatically when you enter the shell. It is not global though. Maybe you create your own wrapper for IDEs though (with direnv exec…)


#3

I have had luck with this setup:


#4

I use this dirty hack to save my direnv “use nix” environments from being garbage collected:

savedrv () {
  if [ "$out" ]
  then
    drv="$(nix show-derivation "$out" | perl -ne 'if(/"(.*\.drv)"/){print$1;exit}')"
    if [ "$drv" ] && [ -e "$drv" ]
    then
      ln -fs "$drv" .drv
      ln -fs "$PWD/.drv" "/nix/var/nix/gcroots/per-user/$LOGNAME/$(basename "$PWD")"
    fi
  fi
}

Works fine with shell.nix, default.nix and also “use nix -p …”. Comments welcome, I don’t know a more elegant way to get the derivation from $out than by show-derivation.


#5

shell.nix can be a little easier to write by hand with mkShell https://github.com/NixOS/nixpkgs/blob/37f50c15da53fbb017a1136049bb11dd8280eed6/doc/shell.section.md


#6

Have a look at the direnv wiki, there are a few similar attempts: https://github.com/direnv/direnv/wiki/Nix


#7

My setup is usually like this:

shell.nix:

{ nixpkgs ? import (fetchGit {
  url = https://github.com/NixOS/nixpkgs-channels;
  ref = "nixpkgs-unstable";
}) {} }:
with nixpkgs;
mkShell {
  buildInputs = [ ... ];
  SOME_ENV_VAR = "foo";
  shellHook = ''
    # usually link in a nix-built vendor directory or other housekeeping
  '';
}

for the already mentioned direnv, my super sophisticated .envrc:

use nix
# optionally also some env vars, if they're supposed to be secret, otherwise they go into `shell.nix`.

My editor of choice is spacemacs, and I’m using emacs-direnv to automatically load my nix-shell for each buffer so all dependencies are found.

For caching the nix-shell I’m using the persistent cached shell variant of use nix, but that’s pretty optional, just speeds things up a lot.

Depending on who I’m working with, I might try to keep all other nix things in a nix/ directory in the project, but I think having a shell.nix and default.nix at top-level is just good practice and easier to work with. So in the subdir go things like a common pinned nixpkgs version, dockertools, overlays, gemset.nix, etc…

To be honest I’m still experimenting with what I find the most comfortable project layout to work with, but after a few years this has been pretty much the standard I build on.


How do I build a nix-shell which depends on some unstable packages?
#8

I had read them, they did not handle the “nix-shell -p” case and I did not want to fiddle with nix-instantiate to duplicate what nix-shell already does.


#9

You don’t have to add that { cargoEnv = foo; } bit – just foo is enough, i.e. the shell.nix file evaluates just to a mkDerivation call directly (with with and optionally let parts before it).

For me creating those shell.nix files just works; I always call nix-shell --pure -Q in that directory. I assume it depends on the particular projects and desired workflows.


#10

I actually sort faced the same issues and thus built myself a small and very opinionated utils: https://github.com/kampka/nixify

It is basically a tool to bootstrap your nix-shell development environment with autoenv support and templates.

It may address some of your issues, but I have to admit that documentation is poor.
You should have a look at how the templates are set up in source code and see if that works for you.

It should at least cover your first two points in the list (maybe 4), but I only tested it with zsh and https://github.com/zpm-zsh/autoenv/ but I’m guessing it’ll work with other shells / autoenvs / direnvs just as well (after bug fixing … )

The tool was built for myself, but I’m happy to look at any issues / pull requests.

Hope this helps :slight_smile:


#11

This is really helpful. The part that I resonate with is

Roughly, I found a tension between the style nixpkgs expects and the style conducive to development

Yes, I don’t think there’s a super simple pathway here. I actually think it needs it’s own topic! Basically there’s no clear “this is what you add to your repo, and he’s how it ends up in a nixpkgs PR”

I don’t think there is a single example of projects that people have nix expression in their repository, and are also contained in nixpkgs. Why? Surely it would make sense. Do you have any examples?


#12

I transformed it in an Org Mode gist file: