Infinite loop when importing fetched Nix expression with relative path

When I import this Nix expression locally, it works:

# rel-path.nix
let
  pwd = ./.;
in
  ''
    ${pwd}
  ''
nix-repl> import ./rel-path.nix
"/nix/store/b22x4cm50gbxl9j6awn7x9z4as19m00j-nix-lang-local-vs-remote-relative-paths\n"

But nix repl becomes unresponsive when importing its fetched equivalent.

  1. First, resetting everything because I ran into issues with nix repl caching results (and nix#1223 issue as well):

    nix-repl> ^D
    
    $ for f in $(sudo find /nix/store -name "*rel-path*"); do nix-store --delete $f; done
    
  2. In a new nix repl:

    nix-repl> import (builtins.fetchurl https://github.com/toraritte/nix-lang-local-vs-remote-relative-paths/raw/main/rel-path.nix)
    

    At this point it can only be stopped with kill -9. Tried it both on Linux and Mac.


The above script is completely useless of course. I came across this while trying to figure out issues with relative paths in one of my Nix expressions, that is sometimes evaluated locally, but other times it is fetched and imported.

builtins.fetchurl https://github.com/toraritte/nix-lang-local-vs-remote-relative-paths/raw/main/rel-path.nix returns

/nix/store/dpaylkqc0kr0wkzq0rc73pgw05ph67v1-rel-path.nix

Therefore, when you import that file, ./. gets evaluated to /nix/store.
And interpolating that path into a string with "${pwd}" copies it to the nix store.

So this effectively copies your nix store into itself, which will take a very long time if your store is big enough.

If use builtins.fetchTarball instead it will work, because then the rel-path.nix file will be in its own subdirectory, instead of being directly in /nix/store.

1 Like

Thank you! This Stackoverflow thread also provides a good explanation to what is happening behind the scenes when interpolating paths.

(To quote:)

User clever on the #nixos IRC channel explained:

When it happens

The expansion into /nix/store/... happens when you use a path inside ${} string interpolation, for example mystring = "cat ${./myfile.txt}.

It does not happen when you use the toString function, e.g. toString ./myfile.txt will not give you a path pointing into /nix/store.

For example:

toString ./notes.txt == "/home/clever/apps/nixos-installer/installer-gui/notes.txt"
"${./notes.txt}"     == "/nix/store/55j24v9qwdarikv7kd3lc0pvxdr9r2y8-notes.txt"

How it happens

The 55j24v9qwdarikv7kd3lc0pvxdr9r2y8 hash part is taken from the contents of the file referenced by the ./path, so that it changes when the file changes and things that depend on it can rebuild accordingly.

The copying of files into /nix/store happens at the time of nix-instantiate; the evaluation of nix expressions is still purely functional (no copying around happens at evaluation time), but instantiation (“building”) is not.

To make this possible, every string in nix has a “context” that tracks what the string depends on (in practice a list of .drv paths behind it).

For example, the string "/nix/store/rkvwvi007k7w8lp4cc0n10yhlz5xjfmk-hello-2.10" from the GNU hello package has some invisible state, that says it depends on the hello derivation. And if that string winds up as the input to stdenv.mkDerivation, the newly made derivation will “magically” depend on the hello package being built.

This works even if you mess with the string via builtins.substring. See this code of nix for how the context of the longer string is extracted in line 1653, and used as the context for the substring in line 1657.

You can get rid of a string’s dependency context using builtins.unsafeDiscardStringContext.

Where it happens in the nix code

${} interpolation uses coerceToString, which has a bool copyToStore argument that defaults to true:

/* String coercion.  Converts strings, paths and derivations to a
   string.  If `coerceMore' is set, also converts nulls, integers,
   booleans and lists to a string.  If `copyToStore' is set,
   referenced paths are copied to the Nix store as a side effect. */
string coerceToString(const Pos & pos, Value & v, PathSet & context,
                      bool coerceMore = false, bool copyToStore = true);

It is implemented here, and the check for the interpolated thing being a ./path, and the copying to /nix/store, is happening just below:

if (v.type == tPath) {
    Path path(canonPath(v.path));
    return copyToStore ? copyPathToStore(context, path) : path;
}

toString is implemented with prim_toString, and it passes false for the copyToStore argument:

/* Convert the argument to a string.  Paths are *not* copied to the
   store, so `toString /foo/bar' yields `"/foo/bar"', not
   `"/nix/store/whatever..."'. */
static void prim_toString(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
    PathSet context;
    string s = state.coerceToString(pos, *args[0], context, true, false);
    mkString(v, s, context);
}