makeExtensibleAsOverlay

Development
1

I wrote a overlay that lets you have a .extend.
Does anyone have advice for writing this better, or can tell if it’s even correct?

with import <nixpkgs> {};
let
  makeExtensibleAsOverlay = (final: parent: let
    _extend = final: parent: next: let newparent = parent // this newparent // (next final parent); in newparent;
    this = parent: {
      extend = _extend final parent;
      }; 
    in this parent);
in 
  lib.fix ((lib.flip makeExtensibleAsOverlay) { o = 0; })

nix-repl> ((import ./meao.nix).extend (self: super: { a = 1; })).extend (self: super: { b = 2; })                                              
{ a = 1; b = 2; extend = «lambda @ /tmp/wtf.nix:7:28»; o = 0; }

When I end up writing something like this I usually don’t understand the whole thing overall, only bits and pieces as I logic and trial/error my way into building it up.
Some advice: functions and parametricity are your friend.

2

A terrible thing you can do with this:

with import <nixpkgs> {}; let
  py = overlay: python3.override { packageOverrides = overlay; };
  traceArgs = f: f.override (old: builtins.seq (lib.traceValSeqN 1 old.extraLibs) old);
  withP = (py makeExtensibleAsOverlay).withPackages;
  final = withP (p:
    let
      first = p.extend (self: super: { lmao = 1; });
      second = first.extend (self: super: { lmao = super.lmao + 1; });
    in
     [ second.lmao ]);
in
  traceArgs final

If you can’t see the signal from the noise, the point is that it’s doing extension inside the anonymous function of .withPackages (p: [ ... ]). As opposed to packageOverrides, you can call it multiple times. So it’s part way to a hack for composeable packageOverrides without modifying upstream, however I haven’t found a way (though I didn’t look particularly hard) to access the extensions from the next .withPackages. It’s probably the same first withPackages every time, which would imply you can’t. So this might largely be an exercise in pointlessness.

3

I think there was an issue for this in nixpkgs…

But, if you like overlay experiments, take a look at overlays: (partially) recursive merge by danbst · Pull Request #54266 · NixOS/nixpkgs · GitHub
It allows extending/overriding python packages without packageOverrides:

self: super: {
  _merge_python27 = true;       # can be moved into super overlay
  python27._merge_pkgs = true;  # can be moved into super overlay

  python27.pkgs.pytest = self.python27.pkgs.pytest_37;
}

NBP brainstormed the idea of creating a small module under _merge attribute for each overlay, to guide merge/extension semantics.

4

sidenote; My gripe with packageOverrides is it doesn’t compose.

5

This is not an issue of packageOverrides per se, but the way it is often used.

It could be composable, if everybody would use overrides like this:

let
  pythonPackageOverlay = overlay: attr: self: super: {
    ${attr} = self.lib.fix (py:
      super.${attr}.override (old: {
        self = py;
        packageOverrides = self.lib.composeExtensions
          (old.packageOverrides or (_: _: { }))
          overlay;
      }));
  };

  packageOverlay1 =
    pythonPackageOverlay
      (self: super: {
        p1 = 1;
        p3 = self.p2 + 1;
      }) "python";
  packageOverlay2 =
    pythonPackageOverlay
      (self: super: {
        p2 = self.p1 + 1;
      }) "python";
in
with import ./. { overlays = [ packageOverlay1 packageOverlay2 ]; };
python.pkgs.p3 # = 3
#python.withPackages (p: [p.p3])

This also ensures that the self reference is correct, which is used in many places in the python passthru set.

4 Likes