(Future) of npm packages in nixpkgs?

Development
23

let me do another guess ; )
i guess the problem are child dependencies

the naive approach would be
symlinking from /nix/store to ./node_modules

but when two packages require the same dependency,
but in a different version, this fails,
cos you have no write access to package contents,
and a shallow node_modules gives collision on package name

pnpm solves this by putting every package in a separate folder
for example ./node_modules/.pnpm/pkg1/node_modules/pkg1
which is a hardlink to ~/.pnpm-store

the hardlink is needed to stop npm from following symlinks
but hardlinks are not portable, so we cant use them for nix

pnpm exposes packages to ./node_modules
via symlinks to ./node_modules/.pnpm
which are resolved by node

my motivation for pnpm-install-only:
pnpm is a crazy-complicated piece of software
and we need only a small part of that complexity.
my script is basically a wrapper around
snyk-nodejs-lockfile-parser (supports multiple lockfile formats)
and uses npm to run lifecycle script (npm run postinstall, etc)

1 Like
24

Hmm, confusing that you call it pnpm-install-only and it uses ~/.pnpm but it doesn’t use pnpm-lock.yaml?

25

yes, because snyk-nodejs-lockfile-parser cannot parse pnpm lockfiles
and i have not yet added the workaround, see support pnpm lockfiles

1 Like
26

Really enjoying this discussion. You guys obviously know quite more than me on that topic.
Seems this discussion died during the winter months.
Do we have anything new on this topic so far?

I have a feeling a ‘pnpm’ like approach with linking of node modules could be the right step to solution.

especially i dont like the current way NodePackages are added to nixpkgs.
Its just too much unpredictable going on with node2nix on this scale

27

I think dream2nix project is where efforts are concentrating.

8 Likes
28

+1 on dream2nix, the right approach from the beginning and nodejs has been the focus “1st class” support, due to its complexity.

1 Like
29

Hey everyone, I’ve announced js2nix here Announcing `js2nix` - scale your Node.js project builds with Nix. Please have a look. I think I managed to address all the @sander’s concerns in this project.

@sander, @Profpatsch, @DavHau, as you are the ones who are interested and contributed in that field lot, I would love to hear your feedback.

5 Likes