Declarative wrappers

Development
1

In Nixpkgs we typically create wrappers using the makeWrapper setup hook. We add the hook to the build inputs, and then we run makeWrapper or wrapProgram to create a wrapper, passing arguments for setting/unsetting/prefixing/suffixing environment variables, but also for setting e.g. argv0 or flags to be passed to the executable.

For a while now I’ve been wanting to have declarative wrappers, mainly so it gets easier to perform the wrapping in a separate derivation and get rid of nested wrappers in case we compose wrappers.

The following is an idea of how I would see this work. In a derivation, one would typically write the values passed on for environment variables.

stdenv.mkDerivation {

  wrappers = with lib.wrappers; {
    # prefix $PATH with git
    "bin/*".PATH.value = [ git ];
    "bin/someprog".XDG_DATA_DIRS.value = [ "$out/share" ];
  };
}

Each environment variable has as value an attribute set:

  • value : Value of the environment variable. String or list of strings.
  • valueModifier : Function that is applied to value. Function.
  • action : What to do when creating a wrapper. E.g. whether the value should be set, unset, appended or prepended. Function.
  • onMerge : What should happen in case of a wrapper merge. Function.

I’m not sure whether another level of nesting is needed in order to deal with argv0 and add-flags

wrappers = with lib.wrappers; {
  # prefix $PATH with git
  "bin/*".vars.PATH.value = [ git ];
  "bin/*".add-flags = [ "--verbose" ];
  "bin/someprog".vars.XDG_DATA_DIRS.value = [ "$out/share" ];
  "bin/someprog".argv0 = "someprog";
};

The lib.wrappers library contains helper functions for dealing with these wrappers, including a function for merging wrappers. We also specify defaults for environment variables, e.g.:

  /* The following are default values used throughout Nixpkgs */
  defaults = {
    LD_LIBRARY_PATH = {
      action = set;
      onMerge = prepend;
      valueModifier = makeLibraryPath;
    };
    JAVA_HOME = {
      action = set;
    };
    LUA_PATH = {
      action = set;
      onMerge = prepend;
    };
    PERL5LIB = {
      action = set;
      onMerge = prepend;
      valueModifier = makeFullPerlPath;
    };
    PYTHONHOME = {
      action = set;
#       onMerge = raise "Cannot change PYTHONHOME as it will result in breakage.";
    };
    PYTHONPATH = {
      action = set;
      onMerge = prepend;
      # valueModifier needs to be defined in the package set as its interpreter dependent
    };
    PATH = {
      action = prefix ":";
      onMerge = prepend;
      valueModifier = makeBinPath;
    };
    XDG_DATA_DIRS = {
      action = prefix ":";
      onMerge = prepend;
      valueModifier = drvs: makeSearchPath "share" ":" drvs;
    };
  };

A library function is used to generate the arguments to makeWrapper. The stdenv will be extended to extend the the wrappers argument with the defaults and call makeWrapper. That means pulling makeWrapper into stdenv.

I’ve partially implemented this, and would like to know now what you think of this. Especially, regarding the following two points:

  • an attribute set with defaults;
  • including this in stdenv.
7 Likes
2

I think this is an awesome idea, but I’m not sure it should be added to stdenv (it’s bloated enough as it is). Might be better to add a function addWrappers drv { "bin/*".vars.PATH.value = [git]; } that uses overrideAttrs to do this automatically in postInstall or something.

Also, your lib.wrappers stuff seems a lot like the NixOS module system. Might be better to just use lib.evalModules.

3

This looks nice.

I was recently thinking about declarative wrappers for wrapGAppsHook but that would require declarative wrapping on lower level and implementing it in bash would be pain.

4

Would declarative wrappers not also require our build-support function beeing port from bash back to nix to support this kind of API? I would love to see if wrappers like the python one and wrapGAppsHook to compose better when used together.

5

I agree that, at least at this point in time, it may make more sense to do this outside the stdenv, because it will make it easier to iteratively improve it. Still, I do think this is essentially at the same level as patchShebangs which we do include in the stdenv.

6

I pushed a draft to WIP: Declarative wrappers by FRidh · Pull Request #53046 · NixOS/nixpkgs · GitHub

7

@jtojnar I don’t follow you there. Instead of picking up say $GSETTINGS_SCHEMAS_PATH from it’s build-time value, we would have to set a value with the Nix language.

@Mic92 in my proposal I call makeWrapper. The Python “wrapper code” needs to be broken down. It does essentially two things: injecting code and creating wrappers. Both are there to provide a runtime environment, setting PYTHONPATH and PATH. That’s actually what we want to describe in Nix; what is the environment we want to run the executable in. Whether we then patch or wrap is an implementation detail.

Also, talking about Python, I would like to see:

passthru = {
  provides = {
    pythonModule = [ python ]; # This derivation provides modules for the given Python. Already in Nixpkgs.
  };
  requires  = {
    pythonModules = [ ... ]; # We currently abuse `propagatedBuildInputs` for this, both as in its functionality as using the parameter. In `buildPythonPackage`.
  };
};

This information is what is needed to create wrappers that can also handle nested Python environments.

I also think this approach would be very useful throughout Nixpkgs.

1 Like
8

We have no efficient way to check if a package contains schemas at evaluation time.

We would need to declare them in passthru or something. Same for every other wrap-gapps-hook variable.

9

We would need to declare them in passthru or something. Same for every other wrap-gapps-hook variable.

Exactly. I think that’s the right direction, though it may be more effort.

10

What I do not like here is the fragment of BASH-code in value.

First, it is not obvious where to write "$out/share" and where "$(out)/share" without looking at the surrounding BASH-code (reading is even more harder)

Second, someone definetively will write tricky derivations with many lines of BASH code in value, which will be very hard to read/modify/refactor.

Third, a single place of if stdenv.hostPlatform.isWindows then ... else ... scatters to many places - to each value ($out vs. %out% vs. $ENV{out}) and to each delimiter usage (":" vs ";")

11

I’ve recently started playing with the idea of exposing {whatever, ...}@args: in passthru for everything I write, but I haven’t written enough nix recently to see if there’s a point.

The driving thought is that I shouldn’t need to duplicate code to override things, and this is my reaction to people having lots of inaccessible let expressions. It may be interesting to expose the entire un-drv-ed mkDerivation argument set. Nix is lazy so it should be fine right?

Other side of the horse?