Thanks a lot everyone, this helps a lot. I initially thought by-name was used to improve the efficiency of nix as it is not needed anymore to scan the whole nixpkgs’s repository to find the package.
I definitely agree, and it is important to get it right, so IMO, we should better think longer but come up with an elegant solution. Is there any place where people are discussing these design choices?
Here are some random ideas to find a solution to the above mentioned problems.
Solving the problem of multiple versions
I don’t know how you plan to implement the pkgs.hello.version "2.0", but the most natural idea that comes to my mind to implement this would be to put the following dictionary in either:
- a file
package-configuration.nix containing { flavors = { … }; } where … is described below,
- or directly inside the
mypackage.meta.flavors field; this might be more elegant as we do not need to create a new file, but the downside is that it is harder to get the list of flavors as you first need to use callPackage on the package.nix file to access the meta attribute which might be less efficient… not sure if it matters in practice or not.
The dictionary would contain something like:
rec {
default = {}; # Ideally, the default value is automatically defaulting to {}, just added here for clarity. It could be possible to disable the default by setting "default = null;".
withGui = { enableGtk = true; };
withPluginA = { enablePluginA = true; };
withPluginB = true; # to avoid the common pattern "withXXX = { enableXXX = true; }, we also allow this equivalent notation.
withProprietaryPlugins = {
enablePluginA = true;
enablePluginB = true;
meta.documentation = "Enable all proprietary plugins at once.";
} // withGui ; # <-- this way it is easy to include other flavors by default.
v3 = { version = "3.0;" }; # Not sure if v2 or simply 2 should be used here
}
Then, we would use this to automatically create all flavors of a package, named like:
mypackagemypackage_withGuimypackage_withPluginAmypackage_withPluginBmypackage_withProprietaryPluginsmypackage_v2
This seems reasonably straightforward to implement since:
mypackage_withProprietaryPlugins
is just:
mypackage.override ({default = {};} \\ (import ./package-configuration.nix).flavors).withProprietaryPlugins
(to support the withPluginB = true; syntax, we just need to apply a basic mapAttrs on the dictionary before calling the .withProprietaryPlugins, and we also need to drop the meta field containing the documentation. If we use mypackage.meta.flavors instead of package-configuration.nix, this is trivial to adapt)
so we just need to loop over the ({default = {};} \\ (import ./package-configuration.nix).flavors) dictionary and create all entries (making sure to test if the value is true), seems like a trivial function to write.
To also allow users to enable multiple flavors at once, I would also allow syntax like:
mypackage.withPluginA.withPluginB
(I find this syntax clearer than something like mypackage.withFlavors ["withPluginA", "withPluginB"], and also avoid the problem of adding parenthesis around the expression when including the package in a list), and similarly this should not be too difficult to implement, I think, by simply definining withPluginA as an alias to override ({default = {};} \\ (import ./package-configuration.nix).flavors).withPluginA (I would be surprised if this is not possible in nix language… since it is basically already done by override). I guess we get composition of .withPluginA.withPluginB this way for free since override composes nicely already.
Of course, we should restrict the set of allowed flavor names to always start with with or of the form v1_1_2 to avoid collision with existing names… but this is also very easy to check.
Overall, I think this solution as some benefits:
- simple to implement
- easy to understand, both for nix maintainers and for nix users
- trivial way to document the action of a flavors using the
meta.documentation field of each flavor
- we can even get trivially the documentation of a package from a
nix repl, by simply typing mypackage.meta.flavors (even if we chose to use package-configuration.nix instead of mypackage.meta.flavors, we can still automatically create mypackage.meta.flavors = (import ./package-configuration.nix).flavors;)
- very easy to parse the list of flavors for search engines like search.nixos.org
- composes nicely (we can combine multiple flavors together with the simple syntax described above, if some of them are incompatible, we should raise an error directly in the
mkDerivation)
Drawbacks:
- I’m maybe biased, but I don’t see much…
Solving the problem of stuff using custom callPackage
The problem of packages using custom callPackages like darwin.apple_sdk_11_0.callPackage seems, to me, that we should find a way to avoid this dirty thing, for many reasons. First, it causes obvious issues with the by-name attributed, but even on a more fundamental basis, this composes poorly. I’ll be honest here, I’m not an expert of this topic… and I don’t even know what are all the changes that are needed by such custom callPackages. Are they just providing new packages? Or are they also providing different versions of packages when running darwin, a bit like an overlay ? Or can they do more crazy things? If they just behave like an overlay, providing different version of packages, can’t we create a file like package-configuration.nix like:
{
overlays = [
pkgs.apple_sdk_11_0.overlay
];
}
or, if more complicated things must be applied, we could also introduce decorators (overlays can be seen as a particular case of decorators), which would be functions mapping attrSet -> (attrSet -> derivation) -> derivation, where the attrSet’s are basically the input of the package.nix file? This would allow the decorator to basically apply and transformation of a derivation. In particular, I guess we could write in our package-configuration.nix (this must be confirmed though, I’m not an expert of this):
{
decorators = [
({apple_sdk_11_0, ...}: derivation: apple_sdk_11_0.callPackage derivation {})
];
}
This way, we could combine multiple decorators, for instance if both qt and apple need, somehow, to change the derivation. Or if it is no working, a simple solution may be simply to define a custom callPackage in our package-configuration.nix:
{
# defaults to customCallPackage = { callPackage, ...}: callPackage
customCallPackage = {apple_sdk_11_0, ...}: apple_sdk_11_0.callPackage;
}
but this has the issue of being less composable as we still only allow a single callPackage.
It’s hard for me to say more of this as I don’t know exactly all the possibilities that custom callPackages can do…
