How to use binary cache in NixOS

jheidbrink · December 24, 2019, 11:58am

I’m serving my Nix store as binary cache via nix-serve (installed on Archlinux via Nix-env). I want to use that cache in the demo NixOS Virtualbox appliance. There is an article on binary caches in the NixOS wiki that links to the Nix manual. The Nix manual says I should adapt the substituters line in /etc/nix/nix.conf. However, it looks like I’m not meant to edit that file directly in NixOS. How would I configure the substituters in NixOS?

filalex77 · December 24, 2019, 12:00pm

Take a look at nix.binaryCaches and nix.binaryCachePublicKeys NixOS options.

voland62 · February 11, 2021, 12:25pm

I still confused a bit. NixOS.

I add nix.binaryCaches and nix.binaryCachePublicKeys in /etc/nixos/configuration.nix. Do I need edit /etc/nix/nix.conf for substituters ?
Is there a way to check if binaryCaches actually work? Now I see that instead of downloading ghc from binaryCache, nix build tries to build ghc from sources, so binCache don’t work…
Should I set my current user as trusted user?
Should I run something like “refresh config” after changing /etc/nixos/configuration.nix ?

NobbZ · February 11, 2021, 12:58pm

/etc/nix/nix.conf should be a symlink on nixOS, pointing to a file in the store, generated from what you have set in your /etc/nixos/configuration.nix. Most of the nix.* options affect the /etc/nix/nix.conf, therefore you shouldn’t edit it by hand ever on NixOS.

And as soon as you added the aforementioned options and rebuild/switched, the caches are considered. It should roughly look like this:

github.com

NobbZ/nixos-config/blob/eb2facb4bc46fb245961c387b7146ae58979b226/modules/cachix/nix-community.nix

{
  nix = {
    binaryCaches = [
      "https://nix-community.cachix.org"
    ];
    binaryCachePublicKeys = [
      "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
    ];
  };
}

They work. Though they probably do not have what you need. It is easy to provide a cache (thannks to cachix) and push random builds to it. Though its hard to actually provide recent versions of the builds that match the inputs of recent nixpkgs, and perhaps older nixpkgs as well, sonsidering overlays…

So if you want to use some overlay that pushes to a binary cache, make sure that you use the exact same version of nixpkgs as they used to build and push from the artifacts.

As soon as you have even the slightest input different then they had to populate the cache, then your hash will change and therefore you wont get anything from the cache anymore.

On top of that, nix caches negative lookup for binary caches, so if nix already knows that an output is not available in a cache, it won’t ask that cache again for the next hour or so.

Personally I prefer to have as little trusted users as possible, as a trusted user can change nix configuration on the fly and also they can add unsigned pathes to the store.

Though if you are the only user anyway, I don’t think it can harm much.

Changing /etc/nixos/configuration.nix does nothing, only running nixos-rebuild switch or nixos-rebuild test will actually activate the new configuration. test will make it disappear after a boot, switch wil make it persist the boot, and then there is nixos-rebuild boot which will build and make it the default after next boot, but until then you remain in your previous version of the system configuration.

voland62 · February 11, 2021, 2:58pm

Wow. Thanks to Gods and Nobbz, I’ve done it!
Just nixos-rebuild switch under root makes new config applied. I thought It should be issued only when we install something “system wide”. How I missed it in docs?..

dramforever · February 11, 2021, 9:30pm

/etc/nix/nix.conf is indeed system-wide

You might want to try ~/.config/nix/nix.conf