If I then import nixpkgs with those overlays, in that order, backtrace overrides autoslot, and I can’t access the latter any more.
Note that I’m using final.python310.pkgs.callPackage and not new.callPackage, for example, because I’ve split up the first and second functions (final and prev and new and old) into separate functions, as they share code with other, similar functions.
First, you need to pass a function to override to get access to the previous overridden values.
Then, you can drop old //, since packageOverrides are already plopped over the previous package set.
Finally, you can use lib.composeExtensions to merge the overlay with the previous layer, if present.
overlays =
let
emptyOverlay = final: prev: {};
in
{
autoslot = final: prev: {
python310 = prev.python310.override (prevArgs: {
packageOverrides =
let
ourOverlay = new: old: {
autoslot = new.callPackage ./autoslot.nix {};
};
in
prev.lib.composeExtensions
prevArgs.packageOverrides or emptyOverlay
ourOverlay;
});
};
backtrace = final: prev: {
python310 = prev.python310.override (prevArgs: {
packageOverrides =
let
ourOverlay = new: old: {
backtrace = new.callPackage ./backtrace.nix {};
};
in
prev.lib.composeExtensions
prevArgs.packageOverrides or emptyOverlay
ourOverlay;
});
};
}
Also, I would still go with the new.callPackage. Overlays are complicated enough when they are self-contained – no need to make them even more headache-inducing by mixing the layers.
That did it; thank for all the help! I still want to keep the individual function layers; they’re too useful! Any advice for that? Or is it just that it’s more confusing?
Yes but I fear that if you try that you’re going to cause more problems than you solve.
You seem very interested in how the internals of the overrides and callPackage functions work.
It’s probably worthwhile to spend a few hours experimenting with lib/fixed-points.nix and lib/customization.nix on trivial attribute sets to learn how they work.
Regarding the function layers for multiple instances of overrides, or overriding the python alias? Also, do you have any specific functions in mind I should look at from those two files?
All of nixpkgs runs on fix and extends underneath it all, so the fixed points file you want to understand in its entirety.
Customization uses those fixed points to implement callPackage and makeOverridable which are the ones to focus on there.
Try using fix and extends to implement something like a resolver or crawler to get a feeling for how it works without the complexity of packages and derivations. A practical one for you might be “use fetchurl to recursively lookup dependencies for a package” some registry like pip perhaps. This will show how fix and extends solve an otherwise complex problem. You could solve this with recursion and merging attrsets, but you would perform a lot of redundant fetches or write a DFS toposort which is a mess. When this usage clicks for you, the way that callPackage behaves will probably be demystified.
It should also force you to understand how to control overrides vs overlays which differ in discreet ways. The “controlling layers” approach you mentioned is done with overlays, but knowing how overrides fit into the picture is important. Knowing “when to use each” is at bottom what I’m hoping you get out of the exercise.