Is github going to tighten the noose?

Github just started rate limiting pull and clones from public repositories for unauthenticated requests in Europe. The majority of the nix ecosystem is built upon github. I’ve already been hit by this and used a VPN to do my last update.

Microslop Github might start tightening the noose it has around opensource and once authentication isn’t enough to stop the slop bots they have unleashed on us, paying for access could (maybe?) realiastically be the only way way forward. Docker Hub did it, maybe this is next.

Maybe it’s too early to ring the alarm bells, but maybe, just maybe, we should start thinking of a strategy?

12 Likes

just started rate limiting pull and clones from public repositories for unauthenticated requests in Europe

Isn’t this something they’ve been doing for many years? I remember having problems with it at ZuriHac before the pandemic.

I don’t disagree with your broader point, but it’s not obvious what actually changed here. Tighter limits, just the error code changing, or something else?

6 Likes

What they did before: rate limit API calls
What they do now: rate limit API calls and git access, at least for HTTPS+git.

Seems as if this is yet another win for ssh…

4 Likes

IIRC NixOS org have enterprise contract with GitHub, maybe admin can request gh to allowlist build machine IP addresses to have a higher rate limit without authentication?

If NixOS ever needed a few blocks of IPs I’d be happy to lend them out for free ofc :wink:

For reasons like these, I’ve moved all my Nix projects away from GitHub into Codeberg, and I would invite others to do the same ( https://codeberg.org/viperML )

14 Likes

This isn’t about the Nix Hydra builders, but about end users. Everybody still needs a copy of Nixpkgs, and potentially many more flake inputs. fetchGitHub already uses the API, and updating a flake can prove impossible behind CGNAT. GitHub allows 60 API requests from a single public IPv4 address (they don’t support IPv6 at all); a rate limit which is easily hit behind NAT when updating multiple machines.

I personally am very much in favor of Nix moving its Git infrastructure to a separate Forgejo instance, maybe even providing semi-public signups for adjacent projects (mainly nix-community).

2 Likes

Unauthenticated… So just authenticate. access-tokens is a thing.

Alternatively, use the channel URL as the flake input. nixpkgs.url = "https://nixos.org/channels/nixpkgs-unstable" works for any modern nix (modern means not older than about 18 months!)

5 Likes

Needing a GitHub account for Nixpkgs use doesn’t sound reasonable. Downloading and versioning locally channel tarballs makes sense, though.

13 Likes

Nixpkgs consumption doesn’t depend on Github though. The tarballs can be fetched from channels.nixos.org. I’m aware the Github tarballs are the most common source for flake users, but they probably shouldn’t be (and I personally have replaced them with the zstd tarballs provided by nixos.org in all of my system configs).

16 Likes

I do agree that tarballs from channels are a reasonable thing to use if not making PRs, yes!

I have pinned multipled nixpkgs versions because there is some software on there that I do not want updated (probably there are others too). Does channels.nixos.org keep every commit of nixpkgs?

No, but it keeps historical tarballs that correspond to when the channel actually advanced since that’s how channels work.

7 Likes

Is there a way to map a channel tarball to the Git commit it would’ve been created from?

At the moment https://channels.nixos.org/nixos-unstable redirects to https://releases.nixos.org/nixos/unstable/nixos-26.11pre1068089.c043004d1c69 and that page says nixexprs.tar.zst maps to c043004d1c6985732bcc1cbc5a9c9aecbbb4e0f0

1 Like

I realized you may have been asking something else. The decompressed tarball has a .git-revision file at its root containing the commit it’s from

[juancnuno@usscerritos:/nix/store/zyag7kpddbmjpl5l23n3nqc6a4hcl5pq-source]$ cat .git-revision 
c043004d1c6985732bcc1cbc5a9c9aecbbb4e0f0
1 Like

What they are asking:

Is there an equivalent to github:nixos/nixpkgs/deadbeef, where deadbeef is a commit sha.

2 Likes