Documentation on how the http(s) binary cache API works?

1

Hi, is there any documentation on how the HTTP(s) binary cache API works?
AFAIK you can use GET on /<hash> to get the derivation and GET on /<hash>.ls to get the file listings of the derivation but are these documented anywhere?

5 Likes
2

There’s no specification/documentation, what exactly are you interested in? I can help answer if there are specific questions.

But in general your description is close but incomplete:

  1. Given a /nix/store/<hash>-<name> output store path, before Nix builds it it will ask substituters if they already have a binary available.

  2. Nix will then lookup https://domain/<hash>.narinfo and if it exists, check the signature against given public key and download the nar file from the url in the narinfo.

  3. Nar is unpacked directly into /nix/store and thus avoids the build.

  4. Optionally Hydra also generates .ls for listing contents of the store path and .log for the log how it was built.

Then Nix also supports POST/PUT requests to populate the binary cache via nix copy command essentially uploading the contents as-is.

Note that cachix push performs much faster:

  • bulk query all missing store paths in one request (instead of N requests for N store paths)
  • upload all paths in parallel (nix copy will upload in parallel only when the dependency graph allows that)
6 Likes
Announcing ncps: A Nix Cache Proxy Server for Faster Builds!
3

Can someone put this forward in a README or NixOS documentation ?
(at the very least the NixOS wiki but I much rather official)
:pray:

4

Thanks for the detailed answer. I wish these were documented somewhere…
I wasn’t searching for anything in particular, just being curious about the internal workings of nix.

5

The nix-serve program can be read as a brief reference of the api.

2 Likes
6

Checkout GitHub - fzakaria/nix-http-binary-cache-api-spec: An OpenAPI specification for a Nix HTTP Binary Cache

@domenkozar

7 Likes