How to deal with impure installation scripts in Nix packaging

Help
1

Hello everyone!

I’ve been learning how to create my first nix-packages through unpacked vim plugins. Trying to package this plugin, however, I’ve been having quite a bit of trouble: peek.nvim.

The plugin requires “Deno” as a dependency, and the problem is that in its installation script it uses the fetch() Deno method multiple times to download files. After learning that this is forbidden in the Nix build environment, I got the file through fetchurl as a fixed input in the Nix-expression, then changed with sed one of the fetch() of the installation script (scripts/build.js), and finally made all the necessary adjustments due to the method change.

The above worked, but since the program in its build script does multiple fetch()s, and correcting each one involves modifying many lines of code (as the method itself, and those that apply to the response, change), I was wondering:

  1. Is there any elegant way to deal with correcting these fetch() to convert them to fixed inputs? I thought about cloning the repository and changing the necessary files, but I don’t know if this would be valid in case I want to (for example) do a PR to nixpkgs, and the source is not the upstream.

  2. In general, is there any optimal way to deal with these packages that use ‘impurities’ in their build script? How do the packages in nixpkgs do it?
    (If you have any similar packages that I can study, it would be greatly appreciated).

Thank you very much for your advice!

2 Likes
2

There are usually 2 approaches:

  1. Patching: patch the build scripts to use hardcoded paths to nix fetched things.
  2. Vendoring: put the nix fetched files into a location that the scripts would recognize as a previous download and skip a redownload.

Sometimes you have to mix both approaches, as some tools don’t check for previous downloads.

2 Likes
3

In this particular case, it looks like you can avoid patching by placing the prefetched file where the build script expects it to be.

The details may be a bit different, but something along these lines should work:

let
  gfmCss = fetchurl { ... };
in
stdenv.mkDerivation {
  ...
  postPatch = ''
    cp ${gfmCss} public/github-markdown.min.css
  '';
  ...
}
1 Like
4

Hello again. Sorry for the delay please, it’s been a busy few days. Also many thanks to both of you for your time @Nobbz and @midchildan.

Both options make sense to me, and I think they are the strategies I attempted to implement when I built the derivation. The problem (and what I would like to understand how to tackle using efficiently the strategies you mention to me) are in the following lines of code:

Although the script detects whether the file exists or not, if it does not detect it, it not only gets it, but processes/modifies it (lines 65 to 68). This prevents me from “faking” the fetch by simply placing the file. Even worse, it is not enough to modify only the line that fetches it because the method used is unable to work with local files; but all subsequent methods that apply to the response must also be modified (in my case using sed, because I do not know a better way). This in principle is not too big a problem for a single fetch, the problem is that lines

They run similar scripts, so a successful installation leads to basically changing the entire installation process to use the files previously fetched with Nix.

At this point I think an alternative could be to add to the package a totally different local script written in bash (is it possible to do it in another language?) that performs the whole installation process and ignore the one proposed by the package. My only drawback with this approach is that migrating the installation script to bash can add a lot of complexity to the installation process because it doesn’t have the methods and functionalities offered by deno (hence the question if it can be done in another language, or even if Nix has the capability to do that processing).

I also wonder, is this a good practice if you want to do a PR on Nixpkgs? And, are there cleaner or more automatable options to do so?

Thank you very much for your help!

5

IMO reimplementing the build script should be a last resort, because it greatly increases the maintenance burden of keeping up with upstream. If that ever becomes necessary, changes should ideally be made upstream to ease packaging.

However, in this case, it seems to me that adding a sed command after placing the prefetched file would work just fine.

So, based on my last example:

let
  gfmCss = fetchurl { ... };
in
stdenv.mkDerivation {
  ...
  postPatch = ''
    cp ${gfmCss} public/github-markdown.min.css
    sed -i \
      -e 's/@media (prefers-color-scheme:dark){//' \
      -e '...' \
      public/github-markdown.min.css
  '';
  ...
}
1 Like