Every once in a while someone tries to add packages with generated
node-packages.nix,
which adds on average 10000 Lines of code to nixpkgs.
This is obviously not a great solution but also adding those depdencies to ournodePackagespackage set becomes unbearable because it got very big and slow - regenerating takes almost 1/2 hour even with a fast gigabyte connection.
If your concerns are only about the combined set of node-packages: We
could stop doing that entirely as that even makes reviewing changes
feasible again. Whenever someone proposes to add something to the
“global” list of node packages it can’t realistically be reviewed as the
diff is just too big to judge or even to be displayed on GitHub.
If we are concerned about repository size none of these approaches are
really great as node packages traditionally have tons of dependencies.
Having one node-packages.nix per package in nixpkgs is not great for
keeping eval performance up but it would probably not incur such big
amounts of repository growth. Keep in mind that each modification of any
file will always add another copy of that file to the git object
storage. Updating one large file is therefore likely less efficient in
terms of keeping the repo size down then a set of seldom updated packages.
There has been other alternatives proposed:
- npmlock2nix: read the packages-json.lock → this does not make the amount of code any smaller, the package-json.lock is at least as big as the code generated in
node2nix.
I am biased here (I started the project): While I believe it is the best
approach of packaging anything with just the native package manager it
currently fails in many situations as not every project has started
using lockfiles yet. Asking users to create a lockfile just to package a
program is not a great user experience and even worse: a manual step.
It would work well for all other cases where there is a lockfile as it
doesn’t require code generation and for the most part just passes the
lockfile through to npm.
- fetchNodeModules:
- Advantages: only needs a single checksum, small footprint
- Disadvantages: like buildGoModule or buildRustPackage it may break if npm decides to generate different output, the sharing is not that great between different packages → This addressed be solved by using recursive nix
It might seems like an attractive option as there isn’t much to be done
by the package maintainer except for providing a few hashes. The risk
here is that the fixed outputs will yet again depend on the platform (or
system) that the initial fixed output has was determined on. One of the
situations where it will likely fail is npm install across different
architectures / operating systems (think x86_64-linux vs
aarch64-linux vs aarch64-darwin…). Node packages are famous for
downloading “random” binaries as part of the package installation step.
Those dependencies will not be discovered unless we restrict FODs to
specific platforms. Introducing multiple output hashes (one for
platform) will kill maintainability. I doubt many people have access to
both x86_64 darwin and linux machines and are willing to do that dance
whenever a package has to be bumped.
We’ve had these issues with the go packaging as well as the rust
packaging and they are wasting a lot of time when things go wrong. It
isn’t exactly straight forward to debug these - especially if hydra
already cached the hash which wouldn’t be reproduced on your platform.
- nodePackages with node2nix:
- Advantages: node packages gets shared between different packages
- Disadvantages: Does not scale, packages can not be updated in isolation, big amount of code changes are impossible to review
We can have one node packages set per package as outlined above. It doesn’t
sound great but I believe it is not worse than this approach.
- Ban all node packages from nixpkgs and ask authors to create third-party repos
If we accept subpar packaging and/or repository size is our primary
objective (it shouldn’t be) then that is the right way as many of these
packages have a questionable cost/benefit relationship compared to many
of our core packages.