Idea: Allow caching IFD results inside git repo

1

Background

Whenever you use IFD (Import From Derivation), there are a few things that are slightly worse.

  1. Building is slower, e.g. since the building has to be split into before/after IFD stages
  2. --dry-run doesn’t work, since it can’t figure out what to build before it has built the IFD, so either it would give an incomplete picture or it would do a bunch of work despite claiming to be a “dry run”. (Or the third option, which nix build --dry-run currently choses1: It doesn’t work at all, but gives an error instead.)
  3. Even when the finished product is in a binary cache, users have to download all the dependencies for the IFD and build it in order for nix to figure out what the final product is and that it is already in a cache.

Number 3 is an especially big problem for haskell.callCabal2nix, since the generated nix files are (deliberately?) not put in binary caches.

My Idea

My idea is to have a mechanism for putting the generated nix-files inside the repo, together with a hash of it’s inputs, so that those can be used directly, but still ensuring that they are always up to date, based on the input hashes. When the input hash doesn’t match, it would regenerate the file and if the generated file is identical, add the input hash to the list of allowed input hashes for the file, if the generated file differs, replace it and clear the old input hashes.

This would only make sense when the generated file is platform-independent, but that should almost always be the case when generating nix-files.

It could also be possible to use this for other platform-independent generated files, e.g. generating a .cabal file from a package.yaml file.

Does this idea make sense?

1 Here is an example of the error nix build --dry-run gives:

error: cannot import '/nix/store/l6wja38a4a8rjmvpfyvmjlgqb1z94d2a-cabal2nix-gf-lsp', since path '/nix/store/8zlw373r027cgkc2vd864sn008ya2awx-cabal2nix-gf-lsp.drv' is not valid

       at /nix/store/krf2cv1jcp79f91a5h0d17xp20dyq1nx-nixpkgs-src/pkgs/development/haskell-modules/make-package-set.nix:87:47:

           86|       # info that callPackage uses to determine the arguments).
           87|       drv = if lib.isFunction fn then fn else import fn;
             |                                               ^
           88|       auto = builtins.intersectAttrs (lib.functionArgs drv) scope;
(use '--show-trace' to show detailed location information)
2

I have a sense you might want to participate in the discussion on:

https://www.github.com/Ericson2314/nix-rfcs/tree/plan-dynamism/rfcs%2F0000-plan-dynanism.md

1 Like
3

Where is the discussion on that held?

4

The discussion takes place here: [RFC 0092] Computed derivations by Ericson2314 · Pull Request #92 · NixOS/rfcs · GitHub

1 Like