Use multiple instances of `prev.python.override'

Help
1

Hello!

As in the title; is there a way to have multiple instances of prev.python.override? If I try to put them in a flake overlays set, like so:

overlays = {
    autoslot = final: prev: { python310 = prev.python310.override {
        packageOverrides = new: old: old // { autoslot = final.python310.pkgs.callPackage ./autoslot.nix {}; };
    }; };
    backtrace = final: prev: { python310 = prev.python310.override {
        packageOverrides = new: old: old // { backtrace = final.python310.pkgs.callPackage ./backtrace.nix {}; };
    }; };
}

If I then import nixpkgs with those overlays, in that order, backtrace overrides autoslot, and I can’t access the latter any more.

Note that I’m using final.python310.pkgs.callPackage and not new.callPackage, for example, because I’ve split up the first and second functions (final and prev and new and old) into separate functions, as they share code with other, similar functions.

Thank you kindly for the help!

2

First, you need to pass a function to override to get access to the previous overridden values.

Then, you can drop old //, since packageOverrides are already plopped over the previous package set.

Finally, you can use lib.composeExtensions to merge the overlay with the previous layer, if present.

overlays =
  let
    emptyOverlay = final: prev: {};
  in
  {
    autoslot = final: prev: {
      python310 = prev.python310.override (prevArgs: {
        packageOverrides =
          let
            ourOverlay = new: old: {
              autoslot = new.callPackage ./autoslot.nix {};
            };
          in
          prev.lib.composeExtensions
            prevArgs.packageOverrides or emptyOverlay
            ourOverlay;
      });
    };
    backtrace = final: prev: {
      python310 = prev.python310.override (prevArgs: {
        packageOverrides =
          let
            ourOverlay = new: old: {
              backtrace = new.callPackage ./backtrace.nix {};
            };
          in
          prev.lib.composeExtensions
            prevArgs.packageOverrides or emptyOverlay
            ourOverlay;
      });
    };
  }

Also, I would still go with the new.callPackage. Overlays are complicated enough when they are self-contained – no need to make them even more headache-inducing by mixing the layers.

3 Likes
3

That did it; thank for all the help! I still want to keep the individual function layers; they’re too useful! :sweat_smile: Any advice for that? Or is it just that it’s more confusing?

4

Also, could you possibly help me with overriding the default version of python?

5

I am not sure what you mean.

If by overriding the default version you mean overriding the python3 attribute, it should follow your changes to python310:

6

Never mind, then; got the function layers working. Sorry for the confusion.

And when I said Python, I literally meant the python alias; is there a way to change it to follow python3 or python310 instead of python2?

7

Yes but I fear that if you try that you’re going to cause more problems than you solve.

You seem very interested in how the internals of the overrides and callPackage functions work.

It’s probably worthwhile to spend a few hours experimenting with lib/fixed-points.nix and lib/customization.nix on trivial attribute sets to learn how they work.

1 Like
8

Regarding the function layers for multiple instances of overrides, or overriding the python alias? Also, do you have any specific functions in mind I should look at from those two files?

9

All of nixpkgs runs on fix and extends underneath it all, so the fixed points file you want to understand in its entirety.

Customization uses those fixed points to implement callPackage and makeOverridable which are the ones to focus on there.

Try using fix and extends to implement something like a resolver or crawler to get a feeling for how it works without the complexity of packages and derivations. A practical one for you might be “use fetchurl to recursively lookup dependencies for a package” some registry like pip perhaps. This will show how fix and extends solve an otherwise complex problem. You could solve this with recursion and merging attrsets, but you would perform a lot of redundant fetches or write a DFS toposort which is a mess. When this usage clicks for you, the way that callPackage behaves will probably be demystified.

It should also force you to understand how to control overrides vs overlays which differ in discreet ways. The “controlling layers” approach you mentioned is done with overlays, but knowing how overrides fit into the picture is important. Knowing “when to use each” is at bottom what I’m hoping you get out of the exercise.

2 Likes
10

Got it; thanks for the detailed write-up; I already knew that callPackage works using functionArgs, so this should provide even more clarity!