How does nix choose a substituter when there is more than one choice?

I have set up an intra-company nix cache on an internal CI server.

It provides dependencies much faster than cache.nixos.org as it is local.

My local nix.conf specifies both the internal CI server as well as cache.nixos.org as substituters.

I’ve noticed that the internal CI server is picked for paths that exist on BOTH caches.
I wondered how does it pick a CI server so I tried changing the order of the CI servers in nix.conf but it kept picking the internal CI server.

Does Nix do some latency/speed check to choose a substituter?

Thanks!

After digging through the source, it appears that binary caches are ordered by a priority field, defined in the nix-cache-info file. https://cache.nixos.org/nix-cache-info lists that priority as 40. I’m guessing your internal cache must be defining a lower priority.

It is indeed as you said. The CI server cache’s priority is set to 30.
It must be something’s default value as I’ve never set it myself (I’m using nix-serve)

Thanks for your help!

Looks like nix-serve hardcodes a priority of 30.

It would be nice to have an easily-deployable caching proxy for that kind of scenarios. A lot of companies could benefit from the lower latency and it would save some bandwidth on the public cache.

2 Likes

I can confirm that it would indeed be quite valuable in a company setting :slight_smile: For now we just make sure our cache has high-enough priority.

Note that if you do so, then your cache needs to be reachable from everywhere and at all time.