Looking for clarity on Nix developer shells and their design

Development
1

I’ve been using nix-shell and nix develop for quite some time, mainly to create reproducible development environments in projects (mostly on the JVM) that don’t use Nix otherwise.

Until recently, I didn’t pay much attention to what happens behind the scenes. But over time, I started noticing some unexpected behaviors: temporary files created in nix-specific places, a large number of environment variables (some with multiline values), and escaping issues in VS Code’s integrated terminal (which was recently fixed in this PR).

These quirks made me curious about why these tools behave the way they do. I’ve since read through documentation, experimented with different setups (mkShell, devenv, devshell.nix), and compared how nix-shell and nix develop set up their environments. While I’ve found some explanations, several questions remain - mostly around the reasoning behind these design choices rather than their implementation details.

Hopefully someone can clarify a few of these questions - maybe even paving the way for better documentation or a blog post down the line.

Environment Differences between nix-shell and nix develop

While comparing nix-shell and nix develop, I noticed that they produce surprisingly different rc files and resulting environments. For example:

  • nix-shell unsets the TZ variable, while nix develop keeps it.
  • nix develop handles XDG_DATA_DIRS, which nix-shell ignores.
  • Each exposes a different set of environment variables — for example, NIX_BUILD_CORES, OLDPWD, system and more in one case, versus exitHooks, failureHooks etc in the other.
  • Most of these environment variables aren’t documented, and I couldn’t find a comprehensive list.

Questions

  1. Was this divergence an intentional design change introduced with nix develop?
  2. What’s the expected contract or stability of these environment variables? Are they meant to be relied on, or can they change across Nix releases?
  3. nix develop fails when no value for outputs is provided, while nix-shell does not. Why must outputs be present for nix develop and why isn’t there a fallback for it?

Plain Derivations vs. stdenv-Based Shells

Most projects I’ve seen use mkShell (or mkShellNoCC) to define development environments. In contrast, devshell.nix creates a “naked” shell from a plain derivation, and devenv used to do the same before reverting. The main difference I’ve noticed between plain derivations and those based on stdenv is that the latter supports setup hooks.

Questions

  1. Both nix-shell and nix develop source $stdenv/setup. This behavior is documented explicitly for nix-shell, but is only mentioned indirectly for nix develop. From what I understand, stdenv is primarily a concept defined in nixpkgs, not something that belongs to Nix itself. Was this naming intentional to be “in sync” with nixpkgs or to suggest something broader?
  2. Are setup hooks critical in developer shell environments? My projects mostly use the JVM where static/dynamic linking isn’t a major concern.
  3. What are the practical upsides and downsides of relying on setup hooks to prepare a devshell environment?
  4. Would it make sense for Nix to support a concept of minimal shells that provide only the bare essentials without additional environment variables? Or is this just bikeshedding or even harmful?

Finally, as far as I understand it, neither nix-shell nor nix develop were designed to be used to create general purpose developer shells.

  1. Are these the right Tools for such Developer Shells? Or should we build something similar (ideally in nixpkgs) that solves this differently? Or does this already exists?

I realize that’s a lot of questions, but I’d really appreciate any insights - even partial ones.
If you’ve dug into these differences yourself, have historical context, or opinions on how developer environments should be handled in Nix, I’d love to hear it.

12 Likes
2

Yes.

If they’re documented you can rely on them (NIX_BUILD_CORES is documented for example).

Because you’re choosing to use a flake. Flakes must contain a flake.nix with an outputs attr that is a function of the flake inputs + self and returns an attrset. If you don’t want to use a flake use the -f flag.

Yes, the design is a layering violation.

It can be, for linking in particular.

2 Likes
3

I’d love to read that, for one!

I had similar questions to yours, but not enough conscientiousness to find answers :stuck_out_tongue:

3 Likes
4

Thanks a lot for your response - that already clarifies quite a bit!

Regarding nix develop, I realize my question wasn’t entirely clear.
My flake defines the following outputs:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  };
  outputs = inputs: let
    system = "x86_64-linux";
    pkgs = inputs.nixpkgs.legacyPackages.x86_64-linux;
  in {
    devShells."${system}".default = derivation {
      inherit system;
      name = "minimal-shell";
      builder = "${pkgs.bashInteractive}/bin/bash";
    };
  };
}

However, when I run nix develop for this minimal example, it fails with the following error:

warning: Git tree '/.../minimal-shell' is dirty
error: builder for '/nix/store/3qcdm3094q3qf8zlavyw8ydrjvnk46qf-minimal-shell-env.drv' failed to produce output path for output 'out' at '/nix/store/3qcdm3094q3qf8zlavyw8ydrjvnk46qf-minimal-shell-env.drv.chroot/root/nix/store/sr802w9c3s5mz6pnii2nnqqp0zbdc24h-minimal-shell-env'

One workaround I found is to manually define the outputs variable inside the setup script (similar to what stdenv does here)

# ...
devShells."${system}".default = derivation {
  inherit system;
  name = "minimal-shell";
  builder = "${pkgs.bashInteractive}/bin/bash";
  stdenv = pkgs.writeTextFile {
    name = "minimal-stdenv";
    destination = "/setup";
    text = ''
      : ''${outputs:=out}
    '';
  };
};

So my question is about why nix develop fails when no outputs environment variable is defined when running nix develop - not about the flake’s outputs attribute itself. In other words, why must outputs be present for nix develop, and why isn’t there a fallback value?

5

While digging deeper, I found a few interesting GitHub issues and PRs:

A recurring theme in these discussions is that there are two very different use cases at play:

I guess in general there are two major use cases:

  1. debug a build
  2. create a dev environment with tools suitable for developing on a project

– @DavHau in NixOS/nix#4609

nix develop comes with a bunch of assumptions - it’s designed to provide the build environment of a derivation.
That’s great when you want to poke at a package, rebuild quickly, or debug a derivation.
But those assumptions don’t necessarily match what we need for everyday development work, especially for projects that don’t produce Nix derivations.
And as @zimbatm noted in cachix/devenv#240, maybe nix develop should have been called nix debug, since it’s really about entering a build environment rather than setting up a developer workspace.

This made me wonder whether we’re stretching nix develop beyond what it was designed for - and if there’s any real reason to.
What exactly does Nix core need to do here that can’t be done entirely in userland (e.g. in nixpkgs)?

For debugging, sure - nix develop is convenient.
But for developer environments, I couldn’t find a reason this couldn’t live completely outside of core Nix.

This led me to experiment with a minimal userspace implementation that looks something like this:

pkgs: let
  inherit (pkgs) lib;
  mkRcFile = {
    name,
    shellHook,
    packages,
  }:
    pkgs.writeTextFile {
      name = "${name}-rc";
      text = ''
        [ -n "$PS1" ] && [ -e ~/.bashrc ] && source ~/.bashrc;

        shopt -u expand_aliases

        out='$PWD/outputs/out'
        export NIX_BUILD_TOP="$(mktemp -d -t nix-shell.XXXXXX)"
        ${lib.toShellVar "nativeBuildInputs" packages}
        . "${pkgs.lib.escapeShellArg pkgs.stdenvNoCC}/setup"

        ${shellHook}
        shopt -s expand_aliases
      '';
    };
  mkDeveloperShell = {
    name ? "developer-shell",
    packages ? [],
    shellHook ? "",
    bash ? pkgs.bashInteractive,
    ...
  }:
    pkgs.writeScriptBin name ''
      #!${lib.getExe bash}
      ${pkgs.lib.getExe bash} --rcfile ${mkRcFile {inherit name shellHook packages;}}
    '';
in
  mkDeveloperShell {
    shellHook = ''
      echo $JAVA_HOME
      exit
    '';
    packages = [
      pkgs.jdk_headless
    ];
  }

It gives the same power (including the option to use stdenv with setup hooks) but offers more flexibility - because it can evolve independently of Nix itself.

So here’s the question:
If developer environments can be expressed entirely in userspace, is using nix develop or nix-shell for that purpose an anti-pattern?

1 Like
6

We’ve went down the “naked shell” route in the begining with devenv, but that yields into issues that stdenv hooks don’t get triggered, which almost all packaging in nixpkgs uses.

Even if that design is better, it lead us with a bunch of pain when people were adding packages and things didn’t work.

1 Like
7

Few months ago, I did similar experiments and ended up with app-shell .

1 Like
8

I fully agree that a naked shell is not sufficient to work together with stdenv hooks.

However, I’m not saying that every environment variable in the rc file is bad or unnecessary - some might be redundant, but others are definitely needed (at least for stdenv hooks).

It should be possible to construct almost the same rc file in userland - or at least one that satisfies the needs of stdenv. The example I shared earlier (kept minimal to avoid noise) doesn’t reproduce the nix generated rc file exactly, but it’s enough for at least the Java stdenv hook to execute correctly.

When nix itself generates the rc file, the behavior of nix develop can change between Nix versions, which works against reproducibility. Also, it’s harder to change and customize. If this logic lived inside nixpkgs, it could evolve in sync with stdenv.

That said, I might be overlooking something here - happy to be corrected or shown where this reasoning falls apart.

1 Like