Overlays in nested flakes

Help
1

I’m having trouble getting overlays to work in nested flakes. Let’s say I have a flake named systemFlake, which is a flake describing my system, which uses a flake named flakeB, which in turn needs a flake named flakeC.

flakeC exposes its derivation through an overlay, like so:

{
  outputs = { self, ... }: {
    overlay = final: prev: final.callPackage ./pkg.nix {}; 
  };
}

(pkg.nix is a function that takes some parameters that are present in nixpkgs)

Now I want to use this flake from flakeB, which also exposes an overlay, just like flakeC:

{
  inputs.flakeC.url = "...";
  outputs = { self, flakeC, ... }: {
    overlay = final: prev: final.callPackage ./pkg.nix {}; 
  };
}

This time, the function in pkg.nix also takes a parameter flakeC. But this attr is not in final, unless the caller (systemFlake) includes flakeC in its overlays, which I don’t want. Basically I’d like systemFlake to look like this:

{
  inputs.flakeB.url = "...";
  outputs = { self, nixpkgs, flakeB, ... }:
    let
      pkgs = import nixpkgs { overlays = [ flakeB ]; };
    in {
      nixosConfigurations.foobar.systemPackages = [ pkgs.flakeB ];
    };
}

But this results in an error in flakeB since flakeB’s pkg.nix is called without the required argument flakeC.

I thought of 2 solutions:

  1. Don’t use overlays. I don’t think that’s a good solution since the flakes manpage states that (in the section about follows): “It is worth noting, however, that it is generally not useful to eliminate transitive nixpkgs flake inputs in this way. Most flakes provide their functionality through Nixpkgs overlays or NixOS modules, which are composed into the top-level flake’s nixpkgs input; so their own nixpkgs input is usually irrelevant.”
  2. Use pkgs.extend or pkgs.appendOverlays in flakeB to add flakeC’s overlay to final. I don’t think that’s a good solution since the Nixpkgs manual states that “although it is often preferable to avoid these functions, because they recompute the Nixpkgs fixpoint, which is somewhat expensive to do.”

Do you see other solutions? What’s the recommended way to do this?

1 Like
2

You should use



{
  inputs.flakeB.url = "...";
  outputs = { self, nixpkgs, flakeB, ... }:
    let
      pkgs = import nixpkgs{overlays=[flakeB.overlay];system="x86_64-linux";};
    in {
      nixosConfigurations.foobar=pkgs.nixos{
        environment.systemPackages=[ pkgs.pkgYouWantToInstall ];
      }  ;
    };
}

or, if you want to use nixos’ nixpkgs module

  inputs.flakeB.url = "...";
  outputs = { self, nixpkgs, flakeB, ... }:{
      nixosConfigurations.foobar=nixpkgs.lib.nixosSystem{
       system="x86_64-linux";
       modules=[{
        nixpkgs.overlays=[flakeB.overlay];
        environment.systemPackages=[ pkgs.pkgYouWantToInstall ];
      }  ;
    }];
};
}

};

In general flakes require specifying the build architecture and flake-utils is popular thanks to automating this tedious part.
import <nixpkgs>{} works thanks to a builtin that takes the current architecture and is not aviable inside flakes

1 Like
3

I usually do something like:

let
  overlays = [ flakeA.overlay flakeB.overlay ];
in {
  inherit overlays;
  overlay = lib.composeManyExtensions overlays;
}

That way, an overlay I expose for a flake will expose a flake which has the effects of all of it’s assumed overlays.

2 Likes
4

I guess my problem wasn’t described well enough, sorry about that, as the examples you folks provided don’t fix my problem. I created an example to try to give a better idea of what I’m talking about. The system flake can be found here: https://github.com/sephii/systemFlake

You’ll see it uses the overlay from the flakeB input. The overlay from flakeB, however, needs a package from flakeC, also distributed as an overlay.

flakeB’s flake.nix file looks like this:

{
  inputs.flakeC.url = "github:sephii/flakeC";
  outputs = { self, flakeC }: {
    # FIXME this doesn’t work since flakeC is not provided to ./pkg.nix
    overlay = final: prev: { flakeBPkg = prev.callPackage ./pkg.nix {}; };
  };
}

It should provide flakeCPkg from the flakeC input to pkg.nix, but I don’t know how to do that. Some people on the chat have proposed doing it like so:

{
  inputs.flakeC.url = "github:sephii/flakeC";
  outputs = { self, flakeC }: {
    overlay = final: prev: { flakeBPkg = (prev // flakeC.overlay final prev).callPackage ./pkg.nix {}; };
  };
}

Which doesn’t work (callPackage doesn’t pass the flakeCPkg parameter to ./pkg.nix). Here’s a solution that works, but I’m not sure it’s the “Nix way” to do it and if there’s any downside of doing it like this:

{
  inputs.flakeC.url = "github:sephii/flakeC";
  outputs = { self, flakeC }: {
    overlay = final: prev: { flakeBPkg = prev.callPackage ./pkg.nix (flakeC.overlay final prev); };
  };
}
5

You can use extend or appendOverlays:

    # FIXME this doesn’t work since flakeC is not provided to ./pkg.nix
    overlay = final: prev:
      let
        pkgs = prev.appendOverlays [ flakeC.overlay ];
      in
      { flakeBPkg = pkgs.callPackage ./pkg.nix {}; };

While this is prettier than your solution at the end, it is more costly since the fix-point is recalculated.

A downside is that you need to pass flakeC.overlay final prev to callPackage of each derivation that needs it.

A limitation is that since you are not recalculating the fix-point, dependencies from the new overlay (flakeC) might not be resolved correctly.
E.g.,

  • if you are overwriting derivation A, which is a dependency of derivation B; the derivation B will not use the overwritten derivation A.
  • or, if the new derivation adds multiple new derivations and some of them depends on others, the dependencies won’t be found unless you are using recursive attribute sets.
6

Doesn’t @jonringer’s solution work? Just compose flakeC’s overlay into flakeB’s. This way the final that flakeB sees will include the effects of flakeC.

7

Composing your dependencies’ overlay with your own and providing that leaks internal details. It pollutes your users’ nixpkgs with unwanted attributes in the best case and breaks their derivations in the worst case.

Your users may use a package (vanilla, or overridden by them), say from Nixpkgs, in their derivations that you also change with an overlay of one of your dependencies. appendOverlays only applies those overlays for your derivations so you don’t impose arbitrary changes to your users. This was important for me and from @sephi’s replies also important for them.

8

That’s a good point. In that case I would say that you do in fact need to write (final.extend flakeC.overlay).callPackage ./pkg.nix {}, for the reasons you stated, except note that I’m using final.extend rather than prev.extend. I’m not sure this won’t lead to an infinite recursion! I don’t believe so as this is only used within the value of flakeBPkg, but I’ve never tried extending final like this from within an existing overlay. In any case, you need to extend final instead of prev as otherwise pkgs won’t contain any overlays that come after the current overlay in the final package set.