Need help with building a npm package

Help
1

Hi!
I’m trying to package umami. I used the nodePackages.node2nix script to generate the nix files. Then when building the package it fails when trying to download something.

> geolite2-redist@1.0.7 postinstall /nix/store/1alabyqvd6mibc1mvy1wg5gb0j69wdig-node_umami-0.21.0/lib/node_modules/umami/node_modules/geolite2-redist
> node scripts/postinstall.js

Downloading MaxMind databases from mirror...
events.js:174
      throw er; // Unhandled 'error' event
      ^

Error: getaddrinfo ENOTFOUND raw.githubusercontent.com raw.githubusercontent.com:443
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:56:26)
Emitted 'error' event at:
    at TLSSocket.socketErrorListener (_http_client.js:401:9)
    at TLSSocket.emit (events.js:198:13)
    at emitErrorNT (internal/streams/destroy.js:91:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! geolite2-redist@1.0.7 postinstall: `node scripts/postinstall.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the geolite2-redist@1.0.7 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /build/.npm/_logs/2020-09-07T09_31_29_861Z-debug.log

builder for '/nix/store/kgrs4zmyn4dzlws7sy83za3nx7g448h9-node_umami-0.21.0.drv' failed with exit code 1
error: build of '/nix/store/kgrs4zmyn4dzlws7sy83za3nx7g448h9-node_umami-0.21.0.drv' failed

It looks like npm cannot resolve raw.githubusercontent.com, but it works using nslookup and ping from the same nix-shell session where I execute nix-build -A umami! I think it is this script that fails.

This is my package definition:

{ pkgs, stdenv, nodejs }:
let
  np = import ./composition.nix {
    inherit pkgs nodejs;
    inherit (stdenv.hostPlatform) system;
  };
in
np.umami.override {
  buildInputs = [ pkgs.nodePackages.node-pre-gyp ];
}

Where the composition.nix is generated by node2nix. I tried to add more packages to the buildInputs but without success. Also I don’t know if something is missing or how to find out what could be missing (I’ve never “really” used npm myself…).

Thank you for any advice!

Regards
Eike

Edit:
Using NixOS 20.03.2882.51d115ac89d (Markhor) and nodePackages_10.x.

1 Like
2

geolite2-redist is trying to download something in its postInstall step as you can see here. Since Nix builds are pure you don’t have access to the Internet inside these builds, unless you opt out of the purity (not recommended). Now, to get around this you can do so by manually packaging geolite2-redist since you need to ensure that the postInstall fetched databases are present before the built step, by e.g. moving them where geolite2-redist expects them and removing the postInstall script since this one in particular I think will try to access the Internet and fail either way. At least that’s one way if no one else has a better solution.

3

Ah that makes sense, thank you! So I thought during the build it is ok to download things, like with fetchurl or fetchgit etc. But then this is trying to do it outside the “normal” build phase? I thought this postInstall referred to something in npm. Wouldn’t it be possible to somehow move this step inside the “fetch-source” phase? I would be ok to download this during the build as it is done with sources.

4

I’m a Nix beginner myself but fetchurl and friends are themselves builders and the reason they are allowed to download something is that they require you to pass the SHA in as a parameter. Someone correct me if I’m wrong but this should be a “fixed-output derivation”. Read more about it here for example. In short, this let’s you download something while building, since you specified what the output must look like.

Long story short: I think you’re thinking of “I’ll let this JS code make network requests” but that’s really not ideal in a Nix environment. The ideal would be to handle the package in question separately, so that umami doesn’t need to request it through the network. If it wasn’t late already I’d give it a shot. I hope I remember to try tomorrow.

From your code snippet it looks like it’s really just trying to download some database file. You could try using fetchurl or fetchgit to download this file and then make it available in the builder for umami at whatever path umami would be looking for it.

Sorry for not supplying any code with this but I’ll hopefully try tomorrow :slight_smile:

1 Like
5

Thank you for your reply. I know that it is not possible to just download stuff while building. I was hoping that the magic tool node2nix was somehow moving the downloading part of the npm build into the scope where it is fine :slight_smile: . Using the node2nix tool generated a lot of nix expressions for all the dependencies–I was very impressed! But it seems that npm allows really crazy things during its build (I don’t know npm which is not helpful…). So there has to be some manual fixing.

I guess the problematic dependency must be packaged separately and there the database files need to be downloaded before the build using fetchurl as you described. But having a look at the script, it always does a network request for validating checksums. So it needs to be patched, too, I think. This results in quite some work :slight_smile:

6

I want to see how this is ultimately resolved since it looks like very useful knowledge so I created an issue in the node2nix repository