How to "merge" several derivation outputs for plugin system?

Development
1

I’m working on a package package that supports plugins. It expects them each as a zip file in the res subdirectory of a given path, but only one such ‘global’ path can be given.

Is there a good workflow for “merging” several output directories into one, and wrapping the original executable so I can pass the correct path?

1 Like
2

I think this is what symlinkJoin is used for.

3

How exactly should that be used?

4

In this scenario there are usually two derivations, the package itself and a wrapper derivation that joins the package with the plugins. The general pattern goes like this:

{ stdenv, runCommand... }:
let
  # creates a new derivation that wraps the package binary and attaches to the plugins somehow.
  withPlugins = plugins:
    # some time it's possible to reference all the plugins individually
    # if they all need to live in a folder then use symlinkJoin
    let pluginsRef = symlinkJoin { name = "${pkg.name}-plugins"; paths = plugins; }; in
    runCommand "${pkg.name}-with-plugins" {
      buildInputs = [ makeWrapper ];
      passthru.withPlugins = moarPlugins: withPlugins (moarPlugins ++ plugins);
    } ''
      makeWrapper ${package}/bin/whatever $out/bin/whatever \
        --set PLUGIN_PATH ${pluginsRef}
    '';

  pkg = stdenv.mkDerivation {
    # ...
    passthru = inherit withPlugins;
  };
in
  pkg

Like this it’s then possible to invoke thepackage.withPlugins with a list of plugins and get a new version of the package bound to those plugins.

2 Likes
5

Thank you, that sounds like a possible solution. I’ll try it as soon as I have time!

6

I am also very interested in this approach. I am exploring ways to create a single derivation of R with a large set of R packages so that all the packages are “visible” to R as one package search path pointing to the root of the package “library” instead of each package accessible via its own path/derivation. Are there any examples of using this type of wrapper derivations in nixpkgs?

7

This sounds like how the Haskell packaging works. Are you familiar with that?

8

I am just starting to learn the Haskell ecosystem in nixpkgs. It feels like I need something similar to ghcWithPackages but for R so that I could create a single derivation which would include both R (preferably of a given version) and a given set of packages (leveraging nixpkgs.rPackages for dependencies, etc.) “bundled” together in one R package library. The amount of nix code supporting ghcWithPackages (https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/make-package-set.nix) looks overwhelming though.

9

The general approach is to make a derivation that builds an R package with a parameterised version of the compiler (see haskellPackages.mkDerivation). Then these are stitched together using symlinkJoin (in ghcWithPackages), then the compiler executable is wrapped to point the libdir to the correct place.

A simpler example of all this is the idrisPackages.with-packages function which does the same thing for idris packages.