How to deal with two versions of a package in by-name

Development
10

Agreed, it’s not a pretty fix. It should be possible to avoid having to do that in the future, I’m still working on it (though it’s non-trivial). But yes ultimately pkgs/by-name doesn’t have an answer for this yet.

That should be unaffected. The recommendation is to do this:

  inherit
    ({
      foo_1 = callPackage ../tools/foo/1.nix { };
      foo_2 = callPackage ../tools/foo/2.nix { };
    })
    foo_1
    foo_2
    ;

which still has the attribute/path mappings for grepping.

Yeah. I consider it future work to figure out multiple package versions. There should be a way to make that nice, hopefully also coming with an interface that doesn’t rely on attribute names for version selection, e.g. something like pkgs.hello.version "2.0" or so.

It’s possible to move packages back out of pkgs/by-name if they aren’t defined like <name> = pkgs.callPackage .... While for multiple versions you need some workaround as linked before, if you need darwin.apple_xyz_sdk.callPackage, you can just move it back out of pkgs/by-name.

However I also think that such alternative callPackage’s are something we should try to get away from because of this, and that is already being done for QT, though it’s trickier for the darwin one.

Maybe one of the best explanations is that pkgs/by-name should ideally allow you to upstream a standardised package directory (currently consisting of just package.nix) by just copying it into Nixpkgs, without having to change anything else.

This means that no context other than that package directory is needed to know how the package is defined, which makes it easy to understand, move and edit them. This is also why there’s a check to prevent package directories from referring to files outside them.

The one exception to this right now is that you can still use all-packages.nix to override callPackage arguments. This is because it’s very common to have to add/remove argument overrides often, and it would be annoying to have to move the package back into/out of pkgs/by-name every time for that.

Other changes, such as using a different callPackage, were not added as exceptions because they’re way less common, and generally a one-way street: If you have a package in pkgs/by-name, but you start using pkgs.python3Packages.callPackage, you’ll have to move it outside, but you probably won’t switch back to pkgs.callPackage anyways.

These are both problems that should get fixed better at some point. A promising idea is to use a new standard package directory file like value.nix, which could directly define the value of the attribute, including how package.nix is called, and with which arguments. Something like

# value.nix
pkgs:
pkgs.callPackage ./package.nix {
  python = pkgs.python3;
}

However it’s not clear how value.nix itself should be called. And maybe this is too powerful of a mechanism, because you lose the ability to generically figure out the .override arguments of a package, since it might not even use a standard package.nix file anymore (or maybe that should still be enforced?). It also feels like an opportunity to overhaul the package override interface in a better way, rather than keep relying on callPackage.

Overall this is really non-trivial with a big impact, so it should be thought out carefully. For pkgs/by-name it was decided to not answer all of these questions ahead of time such that we can make smaller incremental improvements.

The one case that wasn’t anticipated is the one for packages with multiple versions. This might be pretty annoying because it is fairly common to add multiple versions. At least with the recommendation, there’s no need to move it back into pkgs/by-name, but it’s not great. Because of that, this seems like the next architectural issue to tackle in Nixpkgs, something like the above-mentioned pkgs.hello.version "2.0" idea.

I don’t think these are a blocker for continuing with pkgs/by-name, because the vast majority of packages only have a single version and pkgs/by-name works well for them. But it’s good to keep these problems in mind. Once I’m done implementing the RFC, I intend to continue working on such problems.

6 Likes