Content addressed buildEnv

Help
1

The docs for documentation.man.man-db.manualPages state that we could provide a content addressed derivation to save some rebuilds if our system packages change but the actual man pages don’t.

Now I looked at the source and the default is

pkgs.buildEnv {
  name = "man-paths";
  paths = lib.subtractLists cfg.skipPackages config.environment.systemPackages;
  pathsToLink = [ "/share/man" ];
  extraOutputsToInstall = [ "man" ]
    ++ lib.optionals config.documentation.dev.enable [ "devman" ];
  ignoreCollisions = true;
}

So how do I make a derivation created by buildEnv content addressed? I only ever find instructions for mkDerivation with the argument __contentAddressed=true. There does not seem to be some official docs for buildEnv in the nixpkgs manual. The closest I could find was a bunch of examples with some short explanation in Declarative Package Management. The functions lua.buildEnv and python.buildEnv seem to be documented much more toroughly :frowning:

To be honest the nixos manual actually warned me (my emphasis):

Advanced users can make this a content-addressed derivation to save a few rebuilds.

So are there any users here who now how to do that? Or is there some hidden documentation for buildEnv that I have missed?

2

Advanced users can make this a content-addressed derivation to save a few rebuilds.

So this description is kind of incorrect. buildEnv will make a derivation whose outputs are symlinks, so they’ll still point to the new builds of the man pages. That means the buildEnv output symlinks will be different, so the content-addressed path for such a derivation would still have to differ, and you’ll get no benefit. You have to make the actual man pages themselves content-addressed, and that would mean making all your environment.systemPackages content-addressed which is very much not advisable.

3

That explains why my experiment with instantiating the above buildEnv call and then running nix store make-content-addressed ./result took forever (I aborted after some minutes).

I do not think that the docs are wrong as the “this” I marked in the option description should refer to the option:

The manual pages to generate caches for if documentation.man.generateCaches is enabled. Must be a path to a directory with man pages under /share/man; see the source for an example. Advanced users can make this a content-addressed derivation to save a few rebuilds.

Still it is a little misleading.

So maybe I should copy the actual man pages to a new derivation with mkDerivation. The following builds just fine, it just takes the default env and then copies all man pages into a new derivation that is made content addressable:

let
  step1 = pkgs.buildEnv {
    name = "man-paths";
    paths = lib.subtractLists [] config.environment.systemPackages;
    pathsToLink = [ "/share/man" ];
    extraOutputsToInstall = [ "man" ]
    ++ lib.optionals config.documentation.dev.enable [ "devman" ];
    ignoreCollisions = true;
    # delete all but english man pages
    postBuild = ''
    find $out/share/man -maxdepth 1 -not -name 'man*' -not -name 'en*' -exec rm -r {} +
    '';
  };
in
pkgs.stdenv.mkDerivation {
  name = "man-paths2";
  src = step1;
  installPhase = "cp -rL $src $out/";
  __contentAddressed = true;
}
4

I’ve settled with this variant:

  documentation.man.man-db.manualPages =
    let
      step1 = pkgs.buildEnv {
        name = "man-paths";
        paths = lib.subtractLists [ ] config.environment.systemPackages;
        pathsToLink = [ "/share/man" ];
        extraOutputsToInstall = [ "man" ]
          ++ lib.optionals config.documentation.dev.enable [ "devman" ];
        ignoreCollisions = true;
      };
    in
    pkgs.runCommandLocal "man-paths-content-addressed"
      {
        src = step1;
        __contentAddressed = true;
      }
      "cp -rL $src $out/";

I’ve omitted removing non english man pages because the savings were negligible (10% on 33mb) also replaced mkDerivation with runCommandLocal to avoid the fixupPhase and the cache query.