# Node\_modules.nix
**URL:** https://discourse.nixos.org/t/node-modules-nix/10004
**Category:** Announcements
**Created:** [November 13, 2020, 9:27am UTC](https://discourse.nixos.org/t/node-modules-nix/10004 "2020-11-13T09:27:15Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@VanCoding](https://discourse.nixos.org/u/VanCoding)
#### Post date: [November 13, 2020, 9:27am UTC](https://discourse.nixos.org/t/node-modules-nix/10004/1 "2020-11-13T09:27:16Z")
Hi guys
I just created a proof-of-concept nix library called “node\_modules.nix”. It also aims at packaging Node.js applications with Nix like node2nix, yarn2nix or pnpm2nix. Like pnpm2nix it’s just a nix-library and not a tool that needs to be run everytime you change your package.json file. But it works directly with NPM.
Check it out here: [GitHub - VanCoding/node\_modules.nix: A nix library to turn your package-lock.json into a fully populated node\_modules directory](https://github.com/VanCoding/node_modules.nix)
I’d be interested in some comments on the idea or suggestions on how to continue.
Thanks 🙂
---
### Author:  [@zimbatm](https://discourse.nixos.org/u/zimbatm)
#### Post date: [November 15, 2020, 9:29am UTC](https://discourse.nixos.org/t/node-modules-nix/10004/2 "2020-11-15T09:29:43Z")
Looking good! I like the simplicity of it.
The base64 conversion could be removed if you can assume a recent-enough version of nix. In that case, it’s possible to use `fetchurl { url = ...; hash = integrity; }` where `integrity` contains the value of the SRI-hash.
Have you tried the tool with `node-sass` yet? Some packages ship with pre-built binaries that need to be patched during the installation. Some other tools want to download them on the Internet during the installation. These are generally the cases that start complexifying the library. I would recommend adding `autoPatchelfHook` to the build as it does a good job at detecting and reporting what libraries are missing.
---
### Author:  [@VanCoding](https://discourse.nixos.org/u/VanCoding)
#### Post date: [November 17, 2020, 5:40pm UTC](https://discourse.nixos.org/t/node-modules-nix/10004/3 "2020-11-17T17:40:27Z")
Thanks for this useful feedback 🙂
Yes, I’m currently experimenting with native modules. My current goal is to bring a gatsby project in our company to work, that uses a lot of native modules for image processing and stuff. That already forced me a tarball-patching feature to library. Wit a few hacks, it’s currently somewhat possible to build the project, but there are still a lot of problems. One of them is that some packages write into the node\_modules folder, which is read-only in this case.
Out of curiosity I tried bringing node-sass to work, and it surprisingly worked pretty well. I added an example for it to the repo.
I’ll add an example where a tarball gets patches soon as well.
---
### Author:  [@zimbatm](https://discourse.nixos.org/u/zimbatm)
#### Post date: [November 18, 2020, 10:53am UTC](https://discourse.nixos.org/t/node-modules-nix/10004/4 "2020-11-18T10:53:32Z")
A common approach is to maintain a list of overrides that map to package names. For example poetry2nix does that. The gem-config for ruby packages in nixpkgs, … That way other users can benefit from the patches and also add their own overrides that they found. Typically that’s used to inject the C dependencies as package managers tend to not handle those.
---
### Author:  [@andir](https://discourse.nixos.org/u/andir)
#### Post date: [November 18, 2020, 11:04am UTC](https://discourse.nixos.org/t/node-modules-nix/10004/5 "2020-11-18T11:04:56Z")
Nice to see this! I’ve worked on something similar for some time:
[GitHub - andir/npmlock2nix](https://github.com/andir/npmlock2nix/) based on this blog post I wrote a while ago: [ranz2nix - A new approach to packaging node packages with Nix · Andreas Rammhold](https://andreas.rammhold.de/posts/ranz2nix/)
I would also like to know how you deal with binaries and NPM packages that try to access the internet during the install stage. I’ve been trying to use `npm ci` as well but those wouldn’t execute any of the NPM hooks. I used those hooks to patchup files (shebangs,symlinks, …) while `npm install` is running as some package might already depend on the scripts/binaries before I get a chance to do.
---
### Author:  [@VanCoding](https://discourse.nixos.org/u/VanCoding)
#### Post date: [November 18, 2020, 11:01pm UTC](https://discourse.nixos.org/t/node-modules-nix/10004/6 "2020-11-18T23:01:24Z")
@zimbatm
This is a good idea! This way, a community could grow around it to add support for packages.
@andir WOW! This takes almost exactly the same approach as my library but already looks way more polished! And there seems to be even someone from Tweag hacking on it. Although it’s quite fun working on node\_modules.nix, it’ll probably lead to me contributing to npmlock2nix in the end 🙂
Today I’ve added a sharp example where I needed to patch a tarball. I think this one takes a bit of a different approach to npmlock2nix, in that it takes the base tarball and modifies it using a provided script. I could be wrong though 😅
That thing with the pre-install hook is also very clever. Are there other differences between npm ci and npm install --offline? I didn’t have the case yet where a shebang had to be patched during installation, though.
---
### Author:  [@andir](https://discourse.nixos.org/u/andir)
#### Post date: [November 18, 2020, 11:54pm UTC](https://discourse.nixos.org/t/node-modules-nix/10004/7 "2020-11-18T23:54:21Z")
> [@VanCoding](#):
>
> WOW! This takes almost exactly the same approach as my library but already looks way more polished! And there seems to be even someone from Tweag hacking on it. Although it’s quite fun working on node\_modules.nix, it’ll probably lead to me contributing to npmlock2nix in the end
That would be neat! Not sure who is continuing the work there but I am keen to work together on improving it! I mostly started adding patches to my fork since I no longer have permissions for the repo in the Tweag Org. One of the experimental features I’ve been working on in Yarn v1 support. That is already working for very simple projects but on some inputs my parser is still tripping over some of the custom format specifics…
> [@VanCoding](#):
>
> Today I’ve added a sharp example where I needed to patch a tarball. I think this one takes a bit of a different approach to npmlock2nix, in that it takes the base tarball and modifies it using a provided script. I could be wrong though
Yeah. I tried to avoid adding patching support to npmlock2nix before actually running into a use case for it. I had it in my mind while working on it but never had the need to use it.
> [@VanCoding](#):
>
> That thing with the pre-install hook is also very clever. Are there other differences between npm ci and npm install --offline? I didn’t have the case yet where a shebang had to be patched during installation, though.
Outside of a Nix sanbox `npm ci` is supposedly better because it should not do network I/O ever. As we have the nix sandbox we do not have to deal with that.
---
### Author:  [@terlar](https://discourse.nixos.org/u/terlar)
#### Post date: [November 20, 2020, 8:26pm UTC](https://discourse.nixos.org/t/node-modules-nix/10004/8 "2020-11-20T20:26:59Z")
This sounds pretty similar to napalm and nix-npm-buildpackage. I think node ecosystem might be the one with most tools now.
Here is a list of the ones I know of:
- [node2nix](https://github.com/svanderburg/node2nix)
- [yarn2nix](https://github.com/nix-community/yarn2nix)
- [yarn2nix by Profpatsch](https://github.com/Profpatsch/yarn2nix)
- [napalm](https://github.com/nmattia/napalm)
- [nix-npm-buildpackage](https://github.com/serokell/nix-npm-buildpackage)
- [nixfromnpm](https://github.com/adnelson/nixfromnpm)
Is my assumption correct? It would be nice if the efforts could be concentrated 🙂
---
### Author:  [@VanCoding](https://discourse.nixos.org/u/VanCoding)
#### Post date: [November 21, 2020, 10:38pm UTC](https://discourse.nixos.org/t/node-modules-nix/10004/9 "2020-11-21T22:38:04Z")
@terlar Nice list. Here some comments on these:
**node2nix**
I already mentioned the drawbacks with this one.
**yarn2nix**
I think it has the same problems as node2nix, but for yarn.
**napalm**
It looks like it starts a local npm registry and installs the packages from there. I didn’t even know that it’s possible to access the local network in the nix sandbox. But since npm supports the cache directory, I think this is unneccessary.
**nixfromnpm**
This tries to solve the problem, that packages are not shared between apps, which is cool. But at the same time, it has the same problem as node2nix and yarn2nix: Before a package can be built, `nixfromnpm` to generate nix or something.
**nix-npm-buildpackage**
Of the mentioned packages, this is the only one that has the same concept as **npmlock2nix** and **node\_modules.nix**. And it looks pretty promising! It also does some clever things to solve problems already mentioned. For example, when running `npm ci`, it uses the `--script-shell` parameter to use a shell, where shebangs are patched before execution. It also has support for yarn, which is nice! One feature it seems to miss, though, is creating a nix-shell containing the node\_modules only.
As I see it, nix-npm-buildpackage, npmlock2nix and node\_modules.nix could be combined into one package, and all 3 packages would benefit from it. Given there is interest, of course.
One additional package that has my special interest is **pnpm2nix**. It uses the brilliant pnpm package manager, which deduplicates packages (like nixfrompnpm). But pnpm currently lacks package hashes, so it’s not there yet.
---
### Author:  [@DavHau](https://discourse.nixos.org/u/DavHau)
#### Post date: [November 23, 2020, 4:23pm UTC](https://discourse.nixos.org/t/node-modules-nix/10004/10 "2020-11-23T16:23:52Z")
Another viable strategy for handling difficult-to-build packages might be to just re-use the node packages from nixpkgs. Even if the lock file wants another package version, you can still take the nixpkgs derivation, override the `src` and `version` attributes and so inherit all the patches and fixes made there.
That way, you avoid building up you own ecosystem of overrides in which you basically just duplicate the work already done in nixpkgs.
Your own collection of overrides will probably never reach the quality of nixpkgs.