I have set up an intra-company nix cache on an internal CI server.
It provides dependencies much faster than cache.nixos.org as it is local.
My local nix.conf specifies both the internal CI server as well as cache.nixos.org as substituters.
I’ve noticed that the internal CI server is picked for paths that exist on BOTH caches.
I wondered how does it pick a CI server so I tried changing the order of the CI servers in nix.conf but it kept picking the internal CI server.
Does Nix do some latency/speed check to choose a substituter?
After digging through the source, it appears that binary caches are ordered by a priority field, defined in the nix-cache-info file. https://cache.nixos.org/nix-cache-info lists that priority as 40. I’m guessing your internal cache must be defining a lower priority.
It is indeed as you said. The CI server cache’s priority is set to 30.
It must be something’s default value as I’ve never set it myself (I’m using nix-serve)
It would be nice to have an easily-deployable caching proxy for that kind of scenarios. A lot of companies could benefit from the lower latency and it would save some bandwidth on the public cache.
For the Chaos Communication Congress i hacked together a module which fulfulls this purpose. Essentially it is just a piece of nginx config, setting the internal hostname as proxy for deployment will cache upstream packages: Sign in · GitLab
in addition to that it seems i have configured apt-cacher-ng as well, 4 years ago: Sign in · GitLab
For now we just make sure our cache has high-enough priority.