NixOS/Flakes dependency on GitHub

Good time to test NixOS resilience to GitHub downtime :slight_smile:

I noticed that nix search nixpkgs something spews few times whole 500 response (including HTML with embedded PNG images) on failed request to get https://github.com/NixOS/flake-registry/raw/master/flake-registry.json.
And nix registry list does the same.
Maybe worth to dump whole response only with some debug env flag.

Is there some way to point Flakes and NixOS to alternative mirrors for such events?

P.S.
nix (Nix) 2.4pre20211006_53e4794
nixos-version: 21.05.20211119.2452847 (Okapi)

The expected approach is to pin registries manually
Options: NixOS Search - Loading...
How I deal with it: flake-utils-plus/options.nix at 1eb995f9885a056a70f2e82738ae29e726e6d43a · gytis-ivaskevicius/flake-utils-plus · GitHub

Why?
By default, the registry is referencing Github repositories and this evaluation is cached for a certain amount of time.
Pinning has these advantages:

  1. You ensure that the required nixpkgs version is always in your /nix/store which means a much faster evaluation time.
  2. You are using the same nixpkgs version as your system configuration does (No need to randomly download 500MB of dependencies when big changes get pushed to nixpkgs)
  3. There are no issues if Github is down :wink:
1 Like

@gytis-ivaskevicius, I use registry to pin some repositories for home-manager wrapper that converts them to -I arguments (NIX_PATH) to fake channels.
That registry have almost no effect on Flakes. They have own pins in flake.lock that may lag behind global registry in /etc/nix/registry.json even if nixpkgs is an implicit input.

My concern is that when GitHub down for a long time or goes away or NixOS namespace blocked - NixOS updates are stalled (think of security patches).
I mostly were seeing errors for fetching https://github.com/NixOS/flake-registry/raw/master/flake-registry.json which is default value for flake-registry in nix.conf.
But I guess it is impossible to update flake.lock for nixpkgs at that time. And I’m not sure what behavior nix flake update have when one of the inputs is not accessible.

I don’t think it will be complete stall. But I expect some hassle to reconstruct repositories, switch to a new source for users and infrastructure that controls branches.

There’s also the problem that GitHub tends to randomly rate limit me from day to day. Fetching nixpkgs often takes 30 to 40 minutes for me. It’s basically unusable. The workaround is logging in using an access token but requiring people to have a GitHub account to use nixos properly seems like a design flaw. I’d prefer if we’d switch back to fastly like we did for channels as this GitHub fetcher is basically unusable for me.

I would expect Software Heritage to fix such issues https://www.softwareheritage.org/ but yeah… We are not there yet

Since it is related, I’ll mention that I’ve already recommended we add a concept of a local flake mirror repository to avoid superflous fetches from github:

I’m hoping I’ll have time to attempt a PR sometime in the next few months :crossed_fingers:

Hosted by Flying Circus.