What is the best dev workflow around nix-shell?

Help
5

shell.nix can be a little easier to write by hand with mkShell https://github.com/NixOS/nixpkgs/blob/37f50c15da53fbb017a1136049bb11dd8280eed6/doc/shell.section.md

6 Likes
6

Have a look at the direnv wiki, there are a few similar attempts: Nix · direnv/direnv Wiki · GitHub

3 Likes
7

My setup is usually like this:

shell.nix:

{ nixpkgs ? import (fetchGit {
  url = https://github.com/NixOS/nixpkgs-channels;
  ref = "nixpkgs-unstable";
}) {} }:
with nixpkgs;
mkShell {
  buildInputs = [ ... ];
  SOME_ENV_VAR = "foo";
  shellHook = ''
    # usually link in a nix-built vendor directory or other housekeeping
  '';
}

for the already mentioned direnv, my super sophisticated .envrc:

use nix
# optionally also some env vars, if they're supposed to be secret, otherwise they go into `shell.nix`.

My editor of choice is spacemacs, and I’m using emacs-direnv to automatically load my nix-shell for each buffer so all dependencies are found.

For caching the nix-shell I’m using the persistent cached shell variant of use nix, but that’s pretty optional, just speeds things up a lot.

Depending on who I’m working with, I might try to keep all other nix things in a nix/ directory in the project, but I think having a shell.nix and default.nix at top-level is just good practice and easier to work with. So in the subdir go things like a common pinned nixpkgs version, dockertools, overlays, gemset.nix, etc…

To be honest I’m still experimenting with what I find the most comfortable project layout to work with, but after a few years this has been pretty much the standard I build on.

6 Likes
8

I had read them, they did not handle the “nix-shell -p” case and I did not want to fiddle with nix-instantiate to duplicate what nix-shell already does.

9

You don’t have to add that { cargoEnv = foo; } bit – just foo is enough, i.e. the shell.nix file evaluates just to a mkDerivation call directly (with with and optionally let parts before it).

For me creating those shell.nix files just works; I always call nix-shell --pure -Q in that directory. I assume it depends on the particular projects and desired workflows.

1 Like
10

I actually sort faced the same issues and thus built myself a small and very opinionated utils: GitHub - kampka/nixify: Bootstrap nix-shell environments

It is basically a tool to bootstrap your nix-shell development environment with autoenv support and templates.

It may address some of your issues, but I have to admit that documentation is poor.
You should have a look at how the templates are set up in source code and see if that works for you.

It should at least cover your first two points in the list (maybe 4), but I only tested it with zsh and GitHub - zpm-zsh/autoenv: Autoenv for zsh but I’m guessing it’ll work with other shells / autoenvs / direnvs just as well (after bug fixing … )

The tool was built for myself, but I’m happy to look at any issues / pull requests.

Hope this helps :slight_smile:

11

This is really helpful. The part that I resonate with is

Roughly, I found a tension between the style nixpkgs expects and the style conducive to development

Yes, I don’t think there’s a super simple pathway here. I actually think it needs it’s own topic! Basically there’s no clear “this is what you add to your repo, and he’s how it ends up in a nixpkgs PR”

I don’t think there is a single example of projects that people have nix expression in their repository, and are also contained in nixpkgs. Why? Surely it would make sense. Do you have any examples?

12

I transformed it in an Org Mode gist file:

4 Likes
13

I like this setup, along with @Mic92’s personal use_nix function. My only suggestion would be to switch the perl command to a jq:

nix show-derivation $out | jq --raw-output 'keys | .[]'
14

I also updated my use_nix based on the latest version in the wiki. The only things I changed is to make it less likely for project names to collide and no dependency on jq.

15

I came to conclusion that nix-build (not nix-shell, because scripts of nix-shell interactive workflow tend to diverge with the derivation code, so a programmer has to maintain both) should have the following command-line options

  1. --nix-build-top=___ to specify the working dir of $NIX_BUILD_TOP
  2. --no-daemon for not sending job to nix-daemon as nixbld1 user will likely have no access to user’s project dir (--nix-build-top=___ might imply --no-daemon)
  3. --phases=___ to choose which phases to perform

Then (in any shell):

  1. Download and unpack sources:
    nix-build --nix-build-top=./ --phases=unpackPhase,patchPhase
  2. Alternative to ./configure (can be run many times with the same working dir)
    nix-build --nix-build-top=./ --phases=configurePhase
  3. Alternative to make (can be run many times with the same working dir reusing .o files)
    nix-build --nix-build-top=./ --phases=buildPhase
  4. Alternative to make test (can be run many times with the same working dir)
    nix-build --nix-build-top=./ --phases=checkPhase
  5. Alternative to make install (requires root privileges, dangerous as it impurely writes the result (which might be manually patched between phases) to nix store; overriding of $out to an user-writable directory can make it safe)
    nix-build --nix-build-top=./ --phases=installPhase,fixupPhase,installCheckPhase
1 Like
16

That would nice, nix-shell is good for setting project-specific environments but it’s not easy to test and debug the building process. But will the dependency to stdenv become a problem？

17

A simple approach to workflow which unfortunately requires a patch https://github.com/NixOS/nix/pull/3036

2 Likes
18

I don’t use NixOS, but I rely on Nix packages on macOS (Darwin), primarily for the isolated development environments use case. I’ve had pretty good luck using the approach of direnv along with a default.nix file in a project directory. It’s a bit slow when switching into the directory, but otherwise tolerable.

Problem: In some projects, I do not want to have Nix developer tools to be available. This happens when the default OS developer tools or environment are actually the ones that should be used to do some project-specific building, and Nix tools get in the way. What should I do if I do not want Nix-specific Clang or anything else to be prepended to the shell’s default path?

Sample default.nix for reference:

with import <nixpkgs> {};

let
  deps = [
    imagemagick
    mysql57
  ];
in
stdenv.mkDerivation {
  name = "my-project";
  src = null;
  buildInputs = deps;
  configurePhase = "";
  buildPhase = "";
}
1 Like
19

I haven’t ever tried this, but I’d be curious to know what happens if you use stdenvNoCC.mkDerivation instead.

1 Like
20

I’m no expert, but as far as I know nix-shell “just” install stuff in
the nix-store and fiddels around with your environment variables.
And if you are using direnv you can manipulate your environment
variables, right after you ran use-nix.

Either you do something like PATH=/usr/bin:$PATH (or where every your
compiler ist) or you fiddel around in you PATH variable and remove the
compiler paths, from the nix-store.
I guess you have to fix more than just the PATH variable.

21

I’ve mixed many ideas from tutorials (like haskell-nix & nix-haskell-monorepo) and approaches posted here, so this is my setup for nix-shell development as of now:

  1. Pin a working version of nixpkgs-channels with the nix-prefetch-git command, save the output as a json file.
$ nix-prefetch-git --rev 8d1510abfb592339e13ce8f6db6f29c1f8b72924 https://github.com/NixOS/nixpkgs-channels.git > .nixpkgs-version.json

Create a pinned-nixpkgs.nix file to load the json file and return our chosen version of nixpkgs:

{ bootstrap ? import <nixpkgs> {}
, json ? ./.nixpkgs-version.json
}:

let
  nixpkgs = builtins.fromJSON (builtins.readFile json);
  src = bootstrap.fetchFromGitHub {
    owner = "NixOS";
    repo = "nixpkgs-channels";
    inherit (nixpkgs) rev sha256;
  };
in
  import src {}
  1. (OPTIONAL) Create a defaul.nix. This one is my default for a simple Haskell project, for Python it would be something like buildPythonApplication. Also, it’s good to use nix-gitignore (or other similar tools) to prevent unwanted files/folders to be part of the nix-build/nix-shell (specially if you plan to use step 4).
{ pkgs ? import ./pinned-nixpkgs.nix {} }:
let
  gitignore = pkgs.nix-gitignore.gitignoreSource [] ./.;
in
pkgs.haskellPackages.callCabal2nix "<project-name>" gitignore {}
  1. Write a shell.nix file that uses your pinned version of nixpkgs, if a default.nix is present, then override it some extra dependencies that might be useful while developing your package.
let
  pkgs = import ./pinned-nixpkgs.nix {};
  dev-pkgs = with pkgs.haskellPackages; [
    cabal2nix
    cabal-install
    ghcid
    stylish-haskell
    hindent
    hlint
  ];
in
(import ./default.nix {}).overrideAttrs (old: {
  buildInputs = old.buildInputs ++ dev-pkgs;
})
  1. (OPTIONAL, but extremely useful) Use direnv with the Persistent cached shell created by @Mic92. This is one easy way to your shell from being garbage collected, you can still delete a cached env by running rm .direnv/env-*.

Change the last line from:

use_nix -s shell.nix -w .nixpkgs-version.json

to

use_nix -s shell.nix

then

$ direnv allow
2 Likes
22

Perfect. That did exactly what I wanted, i.e., kept Nix Clang out of the environment. It still put coreutils, bash, and a bunch of other things on the path, but those have not caused me any grief yet.

Thank you.

1 Like
23

My function for direnv is also maintained in this repository

1 Like
24

you may also be interested in lorri GitHub - target/lorri: Your project's nix-env which is a daemon which will attempt to build your expression in the background. Also I believe it manages it’s own gc-roots, so your less likely to clean your environment.

2 Likes