Why are there so many ways to reference lib and do they matter?

Help
1

In NixOS we have the lib parameter. But also the pkgs parameter that has pkgs.lib. why do we have both? Also I see pkgs.stdenv.lib used in some modules which also seems to be the same lib. Why do we have so many ways to refer to lib?

9 Likes
2

The lib in the NixOS module arguments comes from the module system itself and is the preferred way to reference it in NixOS. It’s the only one that doesn’t depend on pkgs and therefore doesn’t lead to infinite recursion when you use it to define modules themselves (which might define overlays, which can influence pkgs).

The pkgs.lib isn’t related to NixOS. It comes from the fact that pkgs.callPackage automatically passes arguments defined in the pkgs scope. So by having pkgs.lib, you can callPackage files like { lib }: <expression>

And finally, I had to look this up myself, pkgs.stdenv.lib is apparently for convenience, although I’m not entirely sure in which cases it’s more convenient, but this is the reasoning given at its definition site: https://github.com/NixOS/nixpkgs/blob/faf5e8ad8cf2a4b4a09f86b1a6474f1c32479af9/pkgs/stdenv/generic/default.nix#L140-L142

8 Likes
3

stdenv has lib so that way if you just need to toss in a lib function you can write stdenv.lib.foo without touching the arguments list at the top of your file (since every package already includes stdenv).

In fact, all the packages I’ve touched access lib via stdenv, and using meta = with stdenv.lib; { … } seems to be extremely common. I don’t think I’ve actually seen a package yet that pulls in lib as an argument, though surely they exist.

1 Like
4

@volth just did a major clean up where they changed a lot of derivations to using lib directly instead of stdenv.lib: https://github.com/NixOS/nixpkgs/pull/61405

I actually don’t see a reason to use stdenv.lib over lib, it’s more confusing than anything imo. For a long time I thought I had to use stdenv.lib because it had something to do with cross compilation or whatever. We use lib everywhere already, so I’ll argue to just use that over the stdenv one, and I’ll be doing that in future Nix expressions.

4 Likes
5

Your PR appears to be removing the usage of stdenv entirely for the affected files. In that case it absolutely makes sense to use lib directly.

For packages that still are stdenv.mkDerivation it’s less clearly a win to add lib as a top-level package argument. I’m tempted to say writing let inherit (stdenv) lib; in is cleaner because that means there’s no chance of having lib != stdenv.lib inside the package.

I may start using that construct in packages that actually have permanent uses of lib, though I still find stdenv.lib really useful because it lets me do things like insert stdenv.lib.fakeSha256 into my package temporarily without having to modify anything else in the file.

6

Yes, I did change the stdenv.lib → lib (which is actually pkgs.lib because all those files are included using callPackage) together with stdenv.mkDerivation → fetchurl, when the only remaining reason for keeping stdenv was accessing stdenv.lib.
The change is not controversial and, as we see per above, both parties are supporting it.

Nevertheless, IMO stdenv.lib is confusing and it might have sense to deprecate it.

7

The module system is defined in the lib, which include all the functions used to change the definitions such as mkForce and mkIf, as well as all function used to declare options such mkOption or types.

The pkgs attribute on the other hand is computed within the module system, and can be changed by setting options such as nixpkgs.pkgs or nixpkgs.overlay.

Using pkgs.lib instead of lib for function used to manipulate the module system can lead to inconsistencies, such as use of a non-matching version of the module system, and to infinite loops if these options are used to compute the pkgs argument.

3 Likes
8

I’m in favour of deprecating things that lead to problems, so that there’s one good way to do it.

As opposed to having 1 good and 1 bad way and propagating that info via hearsay (or if you’re lucky to read that section, the manual).

I have many times used lib, pkgs.lib, stdenv.lib interchanged, until I read the “lead to inconsistencies” above.

2 Likes
9

I guess another question is why we have stdenv.mkDerivation in the first place. If we want to make all the dependencies explicit it would be best to have it in the top-level like any other mkSomething functions.

Before:

{ stdenv, fetchurl }:
stdenv.mkDerivation {
  ...
  meta = with stdenv.lib; {
    ...
  };
}

After:

{ lib, mkDerivation, fetchurl }:
mkDerivation {
  ...
  meta = with lib; {
    ...
  };
}
5 Likes
10

It’s called stdenv.mkDerivation because that function uses a bunch of information defined in stdenv.

11

stdenvNoCC.mkDerivation might be a better candidate for top-level mkDerivation.
stdenv.mkDerivation brings gcc and other C/C++ tools which are needless in many use cases of mkDerivation

2 Likes