Is `readFile ./foo.sh` an IFD?

1

I often do something like this.

    foo = pkgs.writeShellApplication {
      name = "foo";
      text = builtins.readFile ./foo.sh;
    };

Then I recently learnt about Import From Derivation (IFD), and the fact that it might cause a performance penalty during nix evaluation.

The docs says Passing an expression expr that evaluates to a store path to any built-in function which reads from the filesystem constitutes Import From Derivation (IFD), and builtins.readFile was also listed there as one of those functions.

Then I learned that I can pass --option allow-import-from-derivation false to nix build command to make it fail when IFD is detected. But the expression I wrote above were built successfully.

So I want to clarify my understandings and assumptions about what happened.

  1. When I run nix build, the whole repository was copied to nix store before start evaluating.
  2. We didn’t hit the performance penalty, because during evaluation, ./foo.sh is already in nix store, hence nothing to build, and nothing blocks evaluation.
  3. Nix build was smart enough to classify above expression as not blocking evaluation, by looking at the argument passed to builtins.readFile, which was a path data type, not a string.
  4. We are safe from IFD performance penalty as long as we use --option allow-import-from-derivation false, even if we use builtin functions that causes IFD mentioned in docs. In other word, no manual check is necessary to avoid IFD performance penalty as long as we pass the flag.

Please let me know if I understand any of above wrong.

Thank you

2

It’s not an IFD because foo.sh is just a path, not a derivation.

Yes, IFD is disallowed via that option.

1 Like
3

There’s two main issues with IFD:

  • Eval is single-threaded, and
  • Fully parallelised realisation cannot be achieved if eval is blocked on realising some paths

Avoiding reading the contents of a derivation into the nix evaluator (or enforcing this via the mentioned option) prevents the second issue.

However, I also find that the first issue also causes perf penalties, and there is no nix-enforced option to prevent that penalty.

For example, using builtin fetchers (builtins.fetch* functions) is an antipattern, because they are not derivations, but are instead functions that are evaluated - and therefore the fetching happens serially. (The fix for that would be to use fetchers from nixpkgs.)

Evaluating config of the module system, and evaluating nixpkgs, are also expensive operations, and so reducing the number of instances of the module system and the number of instances of nixpkgs are also good steps to reduce eval time. Avoiding a massive number of unused modules will also help (e.g. I avoid adding needless things like home-manager).

There’s nothing that will technically enforce these, you just have to know the limitations of the evaluator and write your code accordingly.

1 Like