Future of channels and channels.nixos.org in a Flakes world?

As it stands, it seems that there’s a good chance that Flakes will be the future. Then where would channels and channels.nixos.org go? Here are some of my thoughts:

I feel that less reliance on GitHub for releases might be a good idea. If we decide to leave GitHub some day for some reason, it would be nice if we don’t suddenly leave countless mentions of github:NixOS/nixpkgs lying around.

So maybe we should add a way to refer to channels in a Flake URL? Here’s a possible way that I thought of:

  1. When a user specifies an http/https URL, and it leads to (possibly several) redirects, we instead record the final redirection destination in flake.lock.
  2. When a flake.lock is consulted to download the tarball, the URL in flake.lock is used.

This way specifying:

inputs.nixpkgs.url = "https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz";

Would actually just work, as instead of failing with an invalid hash whenever nixos-unstable updates, it saves the redirected URL which points to a stable version. flake.nix would look something like this:

{
  "nodes": {
    "nixpkgs": {
      "locked": {
        "narHash": "sha256-N1qI50AkeTSBp1ffUCHrcK2re52wrn6euFFGGvFa2iw=",
        "type": "tarball",
        "url": "https://releases.nixos.org/nixos/unstable/nixos-21.05pre269929.ff96a0fa563/nixexprs.tar.xz"
      },
      "original": {
        "type": "tarball",
        "url": "https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz"
      }
    },
    // ...
}

This way, older versions of Nix seeing this new lock file would just behave as if someone used --override-input, and newer versions of Nix seeing the old lock file with the pre-redirection URL would simply migrate it over when nix flake update --update-input is used.

A major concern would be whether this redirection is actually part of the intended interface of channels.nixos.org. That’s mainly why this is isn’t a feature request: It isn’t clear whether this even is a feature. I hope to generate some discussion here first.

In any case, I hope that channels.nixos.org doesn’t go away, if not just for a slightly independent place where nix expressions can be found.

4 Likes

On an unrelated note, it seems that channels.nixos.org redirects to the fixed version releases.nixos.org URL with an HTTP 301 instead of a 302. Is this right? There’s no Cache-Control header in the second response…

Trace is abridged from curl -IL https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz:

HTTP/2 302
cache-control: public, max-age=0, must-revalidate
content-length: 73
location: https://channels.nixos.org/nixos-unstable/nixexprs.tar.xz

HTTP/2 301 
content-length: 0
location: https://releases.nixos.org/nixos/unstable/nixos-21.05pre269929.ff96a0fa563/nixexprs.tar.xz

HTTP/2 200 
content-type: application/octet-stream
content-length: 16904176

In case anyone is interested, here’s a proof of concept patch that implements this redirection behavior. No promises, no warranty, and in particular I have not tested throughly what would happen if you upgrade from good old nixUnstable to this.

https://github.com/dramforever/nix-dram/blob/176079d8c6dd77d50f3c8ca54bcadc2019a2af4e/nix-patches/nix-flake-http-redirect.patch

I’m guessing the registry is supposed to replace channels, instead of github:NixOS/nixpkgs you can just use nixpkgs which currently points to the same github repo but can be pointed at any other repo without changing flake.nix (flake.lock would need to be updated though).

Oh that seems reasonable.

yes but channels are actually a subset of the git commits: only built and tested commits. So while I agree that flakes would be nicer than channels, actually the flakes would need to point the channel I think?

So maybe the nixpkgs flake should be pointing to https://channels.nixos.org/nixos-21.11/nixexprs.tar.xz? Does that work?

I’m using flake-based NixOS and Home Manager configurations, and I set the nixpkgs-like inputs from nixos-* branches to ensure that they are built and tested on Hydra and that the binary substitutions are available.

This way, I’m literally using the channels implicitly. Is that right?

Maybe for 301 redirects, but not 302.

1 Like

This was my thought too. A 301 is a permanent redirect and so is suitable for recording in the lockfile. A 302 is temporary, and should be treated as temporary. For example, it could be a CDN pointing you at the local edge node and so you want a different node if you’re fetching it on a different network. Or perhaps the edge node it’s pointing at today isn’t even around tomorrow and it needs to resolve to a different URL tomorrow even if you haven’t changed networks.

The downside of course is that https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz is in fact a 302 redirect, as it’s redirecting you temporarily to the current channel head, and will redirect to a different channel head later. So recording 301 redirects doesn’t solve the original issue.

I think the solution to the original issue is to have a flake input option that tells nix to lock the result of a single redirection (modulo 301 redirects) instead of the original URL in the lockfile. This way you can point at a dynamic URL that you expect to redirect to a permalink and explicitly ask Nix to store the permalink instead.

The detailed behavior here should be “follow all 301 redirects. Upon hitting a single 302 redirect, if the option is specified, follow that and resume following 301 redirects. A subsequent 302 redirect is not followed”.

If I understand how the channel process works, they do already. Hydra builds the channel, once the configured jobset passes the commit that was built is then pushed to the appropriate branch as well as published as a channel tarball. By that I mean that e.g. the nixos-21.11 branch is updated at the same time as the nixos-21.11 channel.

My guess is the source for each channel is the equivalently-named release-* branch, e.g. release-21.11 is presumably what feeds into the nixos-21.11* channels. So if you were to point at flake at the release branch you’d get the release channel equivalent of pointing at master (e.g. if you don’t want to wait for hydra).

Of course, given all that, it makes me wonder why the default flake registry doesn’t specify a branch for nixpkgs, thus having it default to master. I would have expected it to default to nixpkgs-unstable so users who don’t customize their flake registry will get the hydra-built packages by default. I think I’ll go file a bug on that. (Edit: Filed as `nixpkgs` entry should default to `nixpkgs-unstable` ref · Issue #16 · NixOS/flake-registry · GitHub, though given the PRs sitting open I’m not sure anyone’s paying much attention to this repo).

4 Likes

I thought this was strange as well. This inspired me to write:

1 Like

I’ve change the flake registry entry for nixpkgs to use nixpkgs-unstable by default instead of master. (nixpkgs: Use nixpkgs-unstable instead of master by edolstra · Pull Request #17 · NixOS/flake-registry · GitHub) This ensures that users get a revision of nixpkgs that has been built and tested by Hydra.

7 Likes
Hosted by Flying Circus.