Enable compression in binary cache

Help
1

Hello, I’ve followed Binary Cache - NixOS Wiki and succesfully setup a personal nix binary cache.

However, I would like to enable compression, and I can’t find info around on how to achieve so:

What I have

$ curl https://mycache.example.com/3m2xp9zfjhcxf7f49w8l4105j093hjrr.narinfo                                          
StorePath: /nix/store/3m2xp9zfjhcxf7f49w8l4105j093hjrr-lwk_cli-0.3.0
URL: nar/3m2xp9zfjhcxf7f49w8l4105j093hjrr.nar
Compression: none   <--------

Desired:

$ curl https://fzakaria.cachix.org/949dxjmz632id67hjic04x6f3ljldzxh.narinfo

StorePath: /nix/store/949dxjmz632id67hjic04x6f3ljldzxh-mvn2nix-0.1
URL: nar/4026897ef85219b5b697c1c4ef30d50275423857cb7a81e138c4b1025f550935.nar.xz
Compression: xz    <-------- or zstd for example
2

The wiki is recommending a relatively inefficient way to store your binary cache because it’s just using nix-serve. This just serves your actual /nix/store directory as a binary cache, and nix-serve doesn’t do any compression of its own. That’s one reason it recommends nginx; you can configure nginx to do on-the-fly HTTP compression.

Realistically though, it’s better to store the cache itself as pre-compressed nars, and that’s not something nix-serve is going to do. You can make a file://-type store which will store compressed nar files instead, and then you can just serve that directory as a file server with nginx. I use hydra to do this, but the way hydra does it is effectively equivalent to

nix copy --to "file:///path/to/cache?secret-key=/path/to/key&compression=zstd" nixpkgs#hello

Note the compression=zstd, which causes nars to be written with zstd. This goes with nginx to serve it:

  services.nginx = {
    enable = true;
    virtualHosts."nixcache" = {
      root = "/path/to/cache";
      listen = [{
        addr = "0.0.0.0";
        port = 3001;
      }];
      default = true;
    };
  };

You can also consider attic, which takes an even more aggressive approach to efficiency with a dedicated server that does a lot of logic to store things efficiently.

3

Thanks, now it’s more clear to me.

While I understand it would be better to store the cache as pre-compressed nars, it seemed a little more complex to me and in case of my personal cache it’s fine to compress on the fly, especially with a fast algo like zstd.

So I went with adding recommendedZstdSettings = true; in nginx and from the logs I see content is compressed.