How to deal with two versions of a package in by-name

Nixpkgs is migrating to moving packages in a by-name folder. But how should we package that come in multiple flavor. For instance, bespokesynth can enable an option to enable a non-free dependency like in:

bespokesynth = darwin.apple_sdk_11_0.callPackage ../applications/audio/bespokesynth {
  inherit (darwin.apple_sdk_11_0.frameworks) Accelerate Cocoa WebKit CoreServices CoreAudioKit IOBluetooth MetalKit;

bespokesynth-with-vst2 = bespokesynth.override {
  enableVST2 = true;

How is such a package is supposed to be mapped in by-name?

This might concern other packages like jackass: init at v0.2.3 by PowerUser64 · Pull Request #241989 · NixOS/nixpkgs · GitHub or any package that comes with a -full version.

The pkgs/by_name/ has a section on this, see “Recommendation for new packages with multiple versions”.

The corresponding PR #292214 contains links to other PRs that put the recommendation into practice.


I don’t think either of those links provide guidance on how one should structure a new by-name multi-version package. They’re explicitly giving advice on how to avoid failing the checks with non by-name packages (see the examples).

In my opinion it would be nice to have a standard pattern that documents how one should structure multi-version packages, but that doesn’t exist yet from what I can tell.

How I’ve solved it for a few packages is keeping the “default” package at by-name/ap/apackage/package.nix and then adding a manual entry for the other version to all-packages.nix. e.g.

apackage_0_1 = callPackage ../by-name/ap/apackage/0_1.nix {};

Though if you don’t want a default, you could add them all manually to all-packages.

1 Like

The pkgs/by-name structure isn’t really made to handle packages with multiple versions, at least yet. So indeed the recommendation is to just not use pkgs/by-name for such packages for now.

@adamcstephens Your approach is interesting, though I also consider it a bit of a hack, because it makes it non-trivial to change the default version. It’s not invalid though :slight_smile:

@tobiasBora Note that your bespokesynth example would need to be refactored to use the standard pkgs.callPackage to be eligible for pkgs/by-name

1 Like

@afh @Infinisil Thanks for the link, but I agree with adamcstephens, this seems more like a dirty fix… The suggested code is, in my opinion:

  • significantly harder to read (and also, it breaks some quick-and-dirty search practicses that I often use, for instance, I use rg "mypackage =" to find in which file a package is defined… this will no longer work)
  • it does not use the by-name structure anyway, suggesting that by-name is missing important concepts.

Isn’t it a bit too early to push for its adoption? In particular, the fact that it is impossible to move a package from pkgs/by-name back to all packages is a significant issue to me. Like what if I start a package, and then realize that it needs multiple versions, and/or a MacOS port, that cannot (to the best of my knowledge) be realized without it?

Actually, I’m wondering: what is the problem that by-name tries to solve?

@adamcstephens that’s maybe the cleaner approach in my opinion… but still feels a bit hacky, as one still needs to read the all-package.nix file… which is precisely what I would not expect from a package in all-packages.nix.

I have also seen the usage of overrideAttrs and override directly in pkgs/by-name, structure like in


Guess it makes sense as well, maybe even cleaner. It just creates really one folder per version, it might be ok, since I don’t know what by-name tries to solve hard to say. Just, my understanding is that it is not possible to reuse code from another byname, like import ../renode/generic.nix as it is outside of the package folder?

I’ve assumed it’s trying to address some common kinds of contribution friction:

  • merge conflicts that crop up regularly when another merge adds/removes/renames/reorders the same section of all-packages.nix
  • having to understand the directory structure before being able to contribute a new package (and potentially getting asked to move it if you misunderstand)
  • having to understand package groupings/order in all-packages.nix (and potentially getting asked to relocate your new package if you misunderstand)
1 Like

Correct, this should fail the checks as it is not allowed.

Agreed, it’s not a pretty fix. It should be possible to avoid having to do that in the future, I’m still working on it (though it’s non-trivial). But yes ultimately pkgs/by-name doesn’t have an answer for this yet.

That should be unaffected. The recommendation is to do this:

      foo_1 = callPackage ../tools/foo/1.nix { };
      foo_2 = callPackage ../tools/foo/2.nix { };

which still has the attribute/path mappings for grepping.

Yeah. I consider it future work to figure out multiple package versions. There should be a way to make that nice, hopefully also coming with an interface that doesn’t rely on attribute names for version selection, e.g. something like pkgs.hello.version "2.0" or so.

It’s possible to move packages back out of pkgs/by-name if they aren’t defined like <name> = pkgs.callPackage .... While for multiple versions you need some workaround as linked before, if you need darwin.apple_xyz_sdk.callPackage, you can just move it back out of pkgs/by-name.

However I also think that such alternative callPackage’s are something we should try to get away from because of this, and that is already being done for QT, though it’s trickier for the darwin one.

Maybe one of the best explanations is that pkgs/by-name should ideally allow you to upstream a standardised package directory (currently consisting of just package.nix) by just copying it into Nixpkgs, without having to change anything else.

This means that no context other than that package directory is needed to know how the package is defined, which makes it easy to understand, move and edit them. This is also why there’s a check to prevent package directories from referring to files outside them.

The one exception to this right now is that you can still use all-packages.nix to override callPackage arguments. This is because it’s very common to have to add/remove argument overrides often, and it would be annoying to have to move the package back into/out of pkgs/by-name every time for that.

Other changes, such as using a different callPackage, were not added as exceptions because they’re way less common, and generally a one-way street: If you have a package in pkgs/by-name, but you start using pkgs.python3Packages.callPackage, you’ll have to move it outside, but you probably won’t switch back to pkgs.callPackage anyways.

These are both problems that should get fixed better at some point. A promising idea is to use a new standard package directory file like value.nix, which could directly define the value of the attribute, including how package.nix is called, and with which arguments. Something like

# value.nix
pkgs.callPackage ./package.nix {
  python = pkgs.python3;

However it’s not clear how value.nix itself should be called. And maybe this is too powerful of a mechanism, because you lose the ability to generically figure out the .override arguments of a package, since it might not even use a standard package.nix file anymore (or maybe that should still be enforced?). It also feels like an opportunity to overhaul the package override interface in a better way, rather than keep relying on callPackage.

Overall this is really non-trivial with a big impact, so it should be thought out carefully. For pkgs/by-name it was decided to not answer all of these questions ahead of time such that we can make smaller incremental improvements.

The one case that wasn’t anticipated is the one for packages with multiple versions. This might be pretty annoying because it is fairly common to add multiple versions. At least with the recommendation, there’s no need to move it back into pkgs/by-name, but it’s not great. Because of that, this seems like the next architectural issue to tackle in Nixpkgs, something like the above-mentioned pkgs.hello.version "2.0" idea.

I don’t think these are a blocker for continuing with pkgs/by-name, because the vast majority of packages only have a single version and pkgs/by-name works well for them. But it’s good to keep these problems in mind. Once I’m done implementing the RFC, I intend to continue working on such problems.


Thanks a lot everyone, this helps a lot. I initially thought by-name was used to improve the efficiency of nix as it is not needed anymore to scan the whole nixpkgs’s repository to find the package.

I definitely agree, and it is important to get it right, so IMO, we should better think longer but come up with an elegant solution. Is there any place where people are discussing these design choices?

Here are some random ideas to find a solution to the above mentioned problems.

Solving the problem of multiple versions

I don’t know how you plan to implement the pkgs.hello.version "2.0", but the most natural idea that comes to my mind to implement this would be to put the following dictionary in either:

  • a file package-configuration.nix containing { flavors = { … }; } where … is described below,
  • or directly inside the mypackage.meta.flavors field; this might be more elegant as we do not need to create a new file, but the downside is that it is harder to get the list of flavors as you first need to use callPackage on the package.nix file to access the meta attribute which might be less efficient… not sure if it matters in practice or not.

The dictionary would contain something like:

rec {
  default = {}; # Ideally, the default value is automatically defaulting to {}, just added here for clarity. It could be possible to disable the default by setting "default = null;".
  withGui = { enableGtk = true; };
  withPluginA = { enablePluginA = true; };
  withPluginB = true; # to avoid the common pattern "withXXX = { enableXXX = true; }, we also allow this equivalent notation.
  withProprietaryPlugins = {
    enablePluginA = true;
    enablePluginB = true;
    meta.documentation = "Enable all proprietary plugins at once.";
  } // withGui ; # <-- this way it is easy to include other flavors by default.
  v3 = { version = "3.0;" }; # Not sure if v2 or simply 2 should be used here

Then, we would use this to automatically create all flavors of a package, named like:

  • mypackage
  • mypackage_withGui
  • mypackage_withPluginA
  • mypackage_withPluginB
  • mypackage_withProprietaryPlugins
  • mypackage_v2

This seems reasonably straightforward to implement since:


is just:

mypackage.override ({default = {};} \\ (import ./package-configuration.nix).flavors).withProprietaryPlugins

(to support the withPluginB = true; syntax, we just need to apply a basic mapAttrs on the dictionary before calling the .withProprietaryPlugins, and we also need to drop the meta field containing the documentation. If we use mypackage.meta.flavors instead of package-configuration.nix, this is trivial to adapt)

so we just need to loop over the ({default = {};} \\ (import ./package-configuration.nix).flavors) dictionary and create all entries (making sure to test if the value is true), seems like a trivial function to write.

To also allow users to enable multiple flavors at once, I would also allow syntax like:

  • mypackage.withPluginA.withPluginB

(I find this syntax clearer than something like mypackage.withFlavors ["withPluginA", "withPluginB"], and also avoid the problem of adding parenthesis around the expression when including the package in a list), and similarly this should not be too difficult to implement, I think, by simply definining withPluginA as an alias to override ({default = {};} \\ (import ./package-configuration.nix).flavors).withPluginA (I would be surprised if this is not possible in nix language… since it is basically already done by override). I guess we get composition of .withPluginA.withPluginB this way for free since override composes nicely already.

Of course, we should restrict the set of allowed flavor names to always start with with or of the form v1_1_2 to avoid collision with existing names… but this is also very easy to check.

Overall, I think this solution as some benefits:

  • simple to implement
  • easy to understand, both for nix maintainers and for nix users
  • trivial way to document the action of a flavors using the meta.documentation field of each flavor
  • we can even get trivially the documentation of a package from a nix repl, by simply typing mypackage.meta.flavors (even if we chose to use package-configuration.nix instead of mypackage.meta.flavors, we can still automatically create mypackage.meta.flavors = (import ./package-configuration.nix).flavors;)
  • very easy to parse the list of flavors for search engines like
  • composes nicely (we can combine multiple flavors together with the simple syntax described above, if some of them are incompatible, we should raise an error directly in the mkDerivation)


  • I’m maybe biased, but I don’t see much…

Solving the problem of stuff using custom callPackage

The problem of packages using custom callPackages like darwin.apple_sdk_11_0.callPackage seems, to me, that we should find a way to avoid this dirty thing, for many reasons. First, it causes obvious issues with the by-name attributed, but even on a more fundamental basis, this composes poorly. I’ll be honest here, I’m not an expert of this topic… and I don’t even know what are all the changes that are needed by such custom callPackages. Are they just providing new packages? Or are they also providing different versions of packages when running darwin, a bit like an overlay ? Or can they do more crazy things? If they just behave like an overlay, providing different version of packages, can’t we create a file like package-configuration.nix like:

  overlays = [

or, if more complicated things must be applied, we could also introduce decorators (overlays can be seen as a particular case of decorators), which would be functions mapping attrSet -> (attrSet -> derivation) -> derivation, where the attrSet’s are basically the input of the package.nix file? This would allow the decorator to basically apply and transformation of a derivation. In particular, I guess we could write in our package-configuration.nix (this must be confirmed though, I’m not an expert of this):

  decorators = [
    ({apple_sdk_11_0, ...}: derivation: apple_sdk_11_0.callPackage derivation {}) 

This way, we could combine multiple decorators, for instance if both qt and apple need, somehow, to change the derivation. Or if it is no working, a simple solution may be simply to define a custom callPackage in our package-configuration.nix:

    # defaults to customCallPackage = { callPackage, ...}: callPackage
    customCallPackage = {apple_sdk_11_0, ...}: apple_sdk_11_0.callPackage;

but this has the issue of being less composable as we still only allow a single callPackage.

It’s hard for me to say more of this as I don’t know exactly all the possibilities that custom callPackages can do…