Possible best practice for using lib.extend for custom config functions

Development Documentation
1

I was recently made aware that the nixpkgs manual recommends against using lib.extend to add custom functions to a config because doing so may cause interoperability conflicts with other modules. While I understand the concern and agree with the intent of the recommendation, I’d like to discuss a potential compromise that would, instead, recommend a best practice when using lib.extend for custom functions.

In brief, the interoperability conflicts occur when lib.extend is used to provide a custom function via lib.foo without the knowledge that another module also relies on a “different” lib.foo. Instead of using lib.extend, the recommendation is to add a separate attribute, via specialArgs, that provides the custom function. In this case you would have a custom function provided through someCustomLib.foo.

Functionally the recommendation above is reasonable, however in practice it is somewhat clunky in cases where both lib and someCustomLib are used in a module.

Consider the following rudimentary but real world example:

{
  inputs,
  lib,
  configLib,
  config,
  pkgs,
  ...
}:
{
  imports = lib.flatten [
    inputs.disko.nixosModules.disko
    (configLib.relativeToRoot "hosts/common/disks/btrfs-disko.nix")
# ...

My suggestion is that instead of recommending against using lib.extend altogether, we could suggest a best practice that custom functions specific to a config be provided through lib.custom.foo so that they are easily distinguished from lib.* functions and unlikely to cause conflicts.

In my own nix-config flake I recently started to do so using:
lib = nixpkgs.lib.extend (self: super: { custom = import ./lib { inherit (nixpkgs) lib; }; });

Note that this approach also allows lib.custom to propagate into hm. See: nixos: use `lib` argument instead of `pkgs.lib` by ThinkChaos · Pull Request #3454 · nix-community/home-manager · GitHub

Using this approach the previous example was refactored to:

{
  inputs,
  lib,
  config,
  pkgs,
  ...
}:
{
  imports = lib.flatten [
    inputs.disko.nixosModules.disko
    (lib.custom.relativeToRoot "hosts/common/disks/btrfs-disko.nix")
# ...

Arguably, the difference is subtle and I may just be biased towards what I see as cleaner looking code, however, in modules where lib and a custom lib are used more heavily the result is more pronounced.

I’d like to get some thoughts from others and potentially submit a PR to update the manual or nix.dev whichever makes the most sense for providing best practices.

References:

The recommendation/warning in nixpkgs manual - Nixpkgs Reference Manual

The relevant commit for that section of the manual, which provides additional context for the recommendation -

1 Like
2

Unfortunately there’s no single best way to wire up a library.
Depending on your needs there may be “easier” ways to achieve things, but all of the “easy” ways get you into trouble when you try to create a module for others to reuse (esp. other codebases).

As a consumer of a such a “third party” module, we shouldn’t have to do anything special besides importing it with imports, and maybe set a flake inputs.foo.follows.
So this precludes any hacks on lib and even specialArgs.
The most versatile way to wire up a library is to use let my-lib = import ./my-lib.nix; in, or in case of flakes, where you may want to reference the scope of the outputs function body:

nixosModules.foo =
  nixpkgs.lib.modules.importApply
    { my-lib = ...; }
    ./foo.nix;

Then write foo.nix to start with

{ my-lib }: { config, lib, ... }:

(i.e. a function returning a module (which is in turn a function))

By doing it this way, you put no unnecessary requirements on your module’s consumer.

This quality of module is not always necessary, but if we teach something else first, we just create a longer, more winding learning curve.

1 Like
3

Hmm, that is unfortunate. I’ll have to consider how your example fits my use cases but agree that reducing the learning curve is desireable.

4

Fascinating use of importApply - I’ve never seen that before.

My personal way of handling a custom lib has been storing it in its own repo, so I can access it across my repos. I make heavy use of a wrapper around lib.packagesFromDirectoryRecursive, automating the typical default.nix seen with packages and lib functions, while still letting different files rely on each other. I like this a lot, but adding custom stuff to specialArgs and the callPackage scope indeed can lead to issues - specifically shadowing. For example, my package collection wrapper was running into shadowing issues when I needed to rely on a local package with the same name as something in nixpkgs under the pkgs scope. I solved this by making a localPackages wrapper around anything local, rather than grabbing directly - but it’s still less safe than multiple stages of function imports. Perhaps I could figure out a way to get that double-wrapper in a callPackage-oriented solution.