Nix Flake "collections"

Help
1

FOLLOW UP: (Probably) Don’t do this. After using this pattern more extensively, I realized that recursively updating locks, or creating giant lists of follows declarations was incredibly tedious.
Instead I recommend that you import flakes where you actually use them - OR USE A REGISTRY. In retrospect I see that what I was doing here, was literally what registries were for, so I was using the wrong tool for the job.

Hey so I have been learning to use flakes a bit more, and started working on converting several of my existing package overlays to be individual flakes.

I am hoping to get feedback critique on my understanding and my approach so far. For context I use several NixOS machines at home with flakes enabled and home-manager setup through NixOS, and at work I run a RHEL box with nix/nixpkgs without flakes or home-manager.

My approach so far was to create individual git repos for each of my packages with a flake.nix, overlay.nix, and default.nix in each of them. Then I made a single git repo that aims to “collect” all of my flakes/overlays into a package set. The goal being that on my NixOS boxes I could add just add the collection to my /etc/nixos/flake.nix and on my RHEL box I could add the overlays. So far I have only really been testing things on my NixOS boxes and am hoping to iron out any quirks there, I imagine I’ll have to get creative with the overlay collection on the RHEL box. Some flake documentation make it seem like the nix flake init -t templates#compat; may be less painful than the old way of managing channels to merge collections so hopefully things won’t be too nasty for the RHEL stuff.

I will just post the snippet from my “collection” flake.nix file, which I am calling ak-nix. So far the collection contains only two packages set_wm_class and ak-core. I “merge” the sub-flakes’ modules and overlays in the ak-nix flake which is really the part of my approach that I am least confident about. Let me know if this seems reasonable :

{
  description = "A collection of aakropotkin's nix flakes";

  inputs.nixpkgs.follows = "nix/nixpkgs";
  inputs.set_wm_class.url = "github:aakropotkin/set_wm_class";
  inputs.ak-core.url = "github:aakropotkin/ak-core";

  outputs = { self, nixpkgs, set_wm_class, ak-core, nix, ... }: {

    packages.x86_64-linux =
      set_wm_class.packages.x86_64-linux //
      ak-core.packages.x86_64-linux;

    overlays.set_wm_class = set_wm_class.overlay;
    overlays.ak-core = ak-core.overlay;
    overlays.ak-nix = final: prev:
      ( set_wm_class.overlay final prev ) //
      ( ak-core.overlay final prev );
    overlay = self.overlays.ak-nix;

    nixosModules.set_wm_class = set_wm_class.nixosModule;
    nixosModules.ak-core = ak-core.nixosModule;
    nixosModules.ak-nix = { pkgs, ... }@args:
      ( set_wm_class.nixosModule args ) //
      ( ak-core.nixosModule args );
    nixosModule = self.nixosModules.ak-nix; 

  };
}

Follow up: this did actually work for my NixOS setups. I’m not sure it’s the intended pattern, but my packages definitely install as expected.

1 Like
2

I am following a similar pattern to this; ultimately to /avoid/ having to create giant lists of follows declarations when developing nixified python packages.

Using mach-nix; in order to keep things sane I try to pin a single version of nixpkgs & mach-nix pypi-deps across all my packages; but this is hard to keep in sync over multiple individual projects with many contributors; so we tend to override the inputs in each package. This ends up with ~ 50 lines of input follows boilerplate in each repo, which is not ideal.

We moved to instead having a central “collection” repo similar to what you described that exposes a standard build function and follows overrides - cutting down all our package imports to around 3 lines, and halving the size of our flake.lock file.

3

Have y’all looked at registries yet?
That was ultimately what I replaced this tangled mess with.
And actually the reason I needed it sounds similar to yours, basically that in an org with a bunch of people, this goofiness of merging and forwarding flakes in large numbers “decentralized” was just not working.

We forked githbub:NixOS/flake-registry, and extended it. The org’s Nix install/update script automatically sets our registry as the system one. In some CI scripts we also take advantage of --registry=<PATH>, or nixConfig.registry = <PATH>; in certain flakes.
It has worked incredibly well; and the ability to “follow X” across a whole collection of flakes with 1 line is incredibly nice compared to the way I had been doing things before.

It might not be the right solution for y’all; but I’d recommend spending ~30 minutes with a few dummy flakes to experiment with it.

Notably: you can even use “non-flake” inputs to registries, as long as you still set inputs.foo.flake = false in your actual flake.nix. Unfortunately ( at least at time of writing ) registries won’t record that field ( I tried it like a dozen ways without any luck ). In our case we used this to act as a “workspace lockfile” that you’ll see in other package managers. This let us split up a giant workspace/mono-repo into separate repositories. Adding things like nix registry --add lodash "https://registry.npmjs.org/.../lodash-X.Y.Z.tgz" --registry ./flake-registry.json; let us lock versions in the same way that a monorepo’s mile long yarn.lock or package-lock.json did.

1 Like