Using github mirrors

1

Hi! Like most nix users (I think), I use github as the authoritative source for package definitions.
However, github seems to have become more and more unreliable lately, and I can’t afford to waste time whenever they happen to be down.

So my questions about this are:

  1. Are there any official mirrors?
  2. I found this mirror which apparently seems to be automated: Dominik Xaver Hörl / nixpkgs · GitLab . How do I tell e.g. fetchTarball to use it as a fallback to github? Maybe some kind of impure env var like this? * Allow the user to override the list of mirrors for specific · NixOS/nixpkgs@76a8d12 · GitHub I can’t seem to find any documentation around this.

Many thanks
Mauricio

2

fetchurl can take urls list instead of a single url string; perhaps support for something like that in other fetchers isn’t good so far.

3

Yeah, for the “main” source of packages (don’t know what the proper name for this is) I could set e.g. NIX_PATH=“nixpkgs=https://gitlab.com/lama-corp/infra/mirrors/nixpkgs/-/archive/nixos-19.09/nixpkgs-nixos-19.09.tar.gz”.
But wherever “github.com” is hardcoded there’s no way to use a mirror. You’d have to manually (or programmatically) actually replace all occurrences.
As you say, it would be great to allow multiple URLs in fetchTarball like fetchurl supports. Think it’s worth a github issue?

4

While reliability of GitHub is definitively becoming an issue (recent experience: my Gist has been removed - not even made private - following an anonymous troll complain), the idea of NixPkgs mirroring unlikely will get a wide support: too many of developers have its copy or fork in their private/local repos, so even disappearance of GitHub wouldn’t affect them

The bigger problem is tons of content referred in fetchFromGitHub…

1 Like
5

This would actually be possible since we can address the content with the fixed output derivation hash. We are currently pushing our tarballs to Software Heritage (nixpkgs-swh | [maintainer=@nlewo]) and fortunately, it is almost possible to fallback on the Software Heritage archive for the fetchFromGitHub fetcher (but there are still some remaining works to achieve…).
There are some details in this blog post also https://www.tweag.io/blog/2020-06-18-software-heritage/.

2 Likes