Relative paths when using `nix-shell` as an interpreter

Development
1

Hi everyone!

At work, we have a huge monorepo and a frozen version of nixpkgs to build services and libraries. There’s a release.nix file containing all our packages and it’s very nice to know that if it can nix-build , then all the services compile and pass unit tests.

I’ve been trying to make our helper scripts reproducible by using nix-shell as a script interpreter. Unfortunately, there seems to be no way of pointing them to our frozen nixpkgs in a nice way. Given the following folder structure:

- nix
  - nixpkgs
    - default.nix # frozen nixpkgs
  - bin
    - helper-script.py

I can write a helper-script.py that looks like this:

#!/usr/bin/env nix-shell
#! nix-shell -I "nixpkgs=." -i python -p "python.withPackages (pkgs: [ pkgs.foo ])"
import foo
foo.blah()

…but it will only work if invoked from the nix directory as ./bin/helper-script.py . It would be really awesome if it was possible to specify pinned nixpkgs relative to the interpreted file if using nix-shell as an interpreter. In this way, no matter how we run that script, it would always resolve nixpkgs to what we have pinned right now.

I looked at the source code of nix-build (nix/nix-build.cc at master · NixOS/nix · GitHub), and there seems no way to make it work now. Do you think adding a feature like that would be useful? How would it look?

…one idea I had is to introduce a new builtin - interpretedFile , that is only available when running nix-shell as interpreter. Since the value of <nixpkgs> is normally a path I want to import, I was wondering if I could make this work by putting a Nix expression in the -I :

#! nix-shell -I "nixpkgs=\"with builtins; (dirOf interpretedFile) + ../nixpkgs/\"" -i python -p "python.withPackages (pkgs: [ pkgs.foo ])"

If dirOf interpretedFile was aliased to something smaller, say <.>, we would be able to write:

#! nix-shell -I "nixpkgs=<.>/../nixpkgs" -i python -p "python.withPackages (pkgs: [ pkgs.foo ])"
1 Like
2

Here are the various ways that I tried, none of them fully satisfactory:

Set the NIX_PATH everywhere

Somehow ensure that NIX_PATH=nixpkgs=path/to/nixpkgs is set in the environment. This has to be replicated for all environment so it can be a bit cumbersome. Using direnv helps a bit.

Exploit the nix-shell interpreter

The nix-shell interpreter load the shell.nix from the script dir shell.nix if no -p argument is passed. This holds true regardless of $PWD. So instead of having:

#!/usr/bin/env nix-shell
#! nix-shell -I "nixpkgs=." -i python -p "python.withPackages (pkgs: [ pkgs.foo ])"

,
write:

#!/usr/bin/env nix-shell
#! nix-shell -i python

and

with import ../nixpkgs {};
mkShell {
  buildInputs = [
    (python.withPackages (pkgs: [ pkgs.foo ]))
  ];
}

The nice thing is that the script now can be invoked from anywhere.

The downside is that if there are many scripts in the same folder, the shell.nix is going to contain all the dependencies and lose it’s precision.

Create a derivation

Instead of using the nix-shell interpreter, generate the script with the pinned dependencies. A quick way to do that is to use pkgs.substituteAll.

The script:

#! @python@/bin/python
import foo

the nix

{ substituteAll, python }:
substituteAll {
  src = ./thescript.py;
  python = python.withPackages (pkgs: [ pkgs.foo ]);
}

The nice thing here is that it’s super precise.

The downside is that @python@ replacements can collide with the real script content. Running the script means first building it, eg: $(nix-build -A myscript)/bin/myscript

3

Expanding on the last section.

Imagine that there is a default.nix in the top-level with all the script derivations defined as attribute sets:

with import ./nixpkgs {}; {
  myscript = callPackage ./scripts/myscript {}
}

Then it’s possible to use nix run to load that environment with all the scripts with precisely-pinned dependencies.

It’s also possible to do something like this: nix-env -f . -ir -p $PWD/.nix-profile and then the script would be available in $PWD/.nix-profile/bin/myscript.

It feels like a better direction but I haven’t fully tried it yet.

4

wrote a (hack) solution https://github.com/NixOS/nix/issues/2470

5

FWIW it does this for any filename argument passed as well, so it doesn’t have to be shell.nix, you can also do

#!/usr/bin/env nix-shell
#!nix-shell deps.nix -i bash

Unfortunately it doesn’t do this for -E flags (NixOS/nix#4232) so you still end up with imprecise dependencies in a single file, or by writing a separate deps file for every script.