Auto-maintaining of hashed mirrors

volth · January 2, 2021, 3:34pm

There are concept of hashed mirrors to backup sources preventing link rot:

NixOS/nix/blob/8a06edbf7e984bae426068189ff9a631bc26af72/src/libstore/globals.hh#L777-L786


      
          Setting<Strings> hashedMirrors{
              this, {}, "hashed-mirrors",
              R"(
                A list of web servers used by `builtins.fetchurl` to obtain files by
                hash. The default is `http://tarballs.nixos.org/`. Given a hash type
                *ht* and a base-16 hash *h*, Nix will try to download the file from
                *hashed-mirror*/*ht*/*h*. This allows files to be downloaded even if
                they have disappeared from their original URI. For example, given
                the default mirror `http://tarballs.nixos.org/`, when building the
                derivation

It is not bot-maintained, so many of the sources are already 404 and not backed up to tarballs.nixos.org.

The most affected are releases which just ended of life (20.03 currently), they are not only stopped receiving security updates, they almost immediately became broken because of link rot of source files which have not been backed up to tarballs.nixos.org.

It might be challenging to create a bot parsing all fetchurl/fetchpatch/fetchgit/fetchFromGitHub (the urls and hashes are often computed), but probably Hydra or OfBorg could assure that everything they download is backed up on the hashed mirrors? That would also reduce the load on upstream websites and tolerate their downtimes.

Introducing something similar for private hashed mirrors would be a great addition too.