I’m probably missing something from the 13th Nix Pill, but when they show
callPackage = path: overrides:
let f = import path;
in f ((builtins.intersectAttrs (builtins.functionArgs f) allPkgs) // overrides);
I don’t understand why it’s not just something like import path (allPkgs // overrides) where the package is required to have variadic arguments?
The way the callPackage pattern is explained it seems like it’s important to first restrict the set of all packages to the ones the function is accepting, but Nix is a lazy language. Wouldn’t passing the whole nixpkgs be the same as passing
builtins.intersectAttrs (builtins.functionArgs f) nixpkgs
or possibly even more efficient, because there is no need to evaluate the intersection (even though lazily), and only the needed part of nixpkgs is accepted by the called function?
On one way I can see how passing in “everything by default” would be considered bad practice, but we’re really doing that anyway, just restricting the “everything” on the side of the caller and not on the side of the callee, even though both are transparent to the user.
2 Likes
That requires every package to be declared as taking variadic arguments. If a package has no need for variadic arguments (and in general they don’t) then why should we put that requirement on the package?
I’m just trying to figure out why the choice was made to be like this. Putting the requirement (or making it part of the design of the language) on the package makes the caller code simpler, which I guess is not that important here.
I still find it surprising that there is an actual call to intersect with the whole package repository, especially if this happens with every single package. It’s probably not a performance issue since something as obvious would be caught early, right?
I do understand that requiring every package to accept variadic arguments is kind of silly. But it also seems silly to have a lazy dynamic language and then do dictionary key restrictions like this, when in reality it could just use laziness and take the parameters it needs out of the set.
Intersection is just as lazy as sets in general are.
In any case, I’m pretty sure sets are strict in the spine (i.e. the set of keys). I don’t know precisely how to prove this though my best attempt at testing by sticking a builtins.trace in the calculation of a set key triggers the moment the set is constructed without even accessing any keys from it.
Given that sets are strict in the spine, an intersection operation isn’t expensive. It just has to collect the set of keys from the smaller set and test for their presence in the larger set.
Even if sets are lazy in the spine, the obvious way to implement intersection given the defined semantics is to collect the keys from the first set and test for their values in the second, and since the first set here is the small set, that means the size of the second set is irrelevant.
Old thread, I know.
One thing I thought of is that if you made a typo and put in a wrong name for a key in overrides, the ‘intersect’ version would catch the error but the ‘variadic’ version would silently ignore the wrong key.