Yarn Plug'n'Play and direnv/packaging

Yes I have been down this rabbit hole for about three months and am about to release a new flake based package manager for Node.js.

I wrestled with Yarn extensively and did have “some success” with PnP; but before you get too excited you need to know - Yarn’s caches and hashing systems are non deterministic and are fundamentally incompatible with Nix’s approach to building.

Frankly Yarn “berry” is poorly implemented, and I found that it produces massive numbers of redundant tarballs because the core hashing system is based on randomly generated seeds. The inline comments explaining the rationale behind randomly generated seeds was based on an incorrect understanding of how filesystem synchronization works ( cough websh*ts cough ). I would strongly suggest that you avoid building large projects with Yarn, and bear in mind that you should not try to include any part of yarn’s global cache in a build output - this will poison your closure. There are notable aspects of their telemetry “feature” which consume an inordinate amount of disk space and time after v2, parts of which appear to be intentionally obfuscated to mask the extent of Facebook’s data collection :frowning: even with telemetry disabled you still suffer the runtime costs of collection, you only prevent reporting ( that is until you invoke yarn with telemetry enabled at which point they send the entire backlog… ). It’s fucking malware in my opinion - and I only say this after digging around its internals for several weeks.

Yarn is more workable with CA hashing; but honestly you’ll be pulling your hair out dealing with the dozens of other poor design decisions and bugs that throw a wrench in reproducible builds.

NPM - much better. It’s got performance issues because of filesystem IO but aside from that I can’t gripe about it too much. Nix’s caching system and built-in fetchers give it a notable performance boost. If you manage a larger project you can cut hours long cache initializations down to a few minutes, with a flexible UX for local dev AND CI which is missing from existing tools.

I’m excited to show off what I’ve been working on soon. But until then I’ll just advise you to “just use NPM with locks and registry packages” if you can, because most existing tools were designed with that in mind. Suffice to say that NPM package-lock.json and flake.lock and “flakeref” URIs are “almost directly compatible” which I’ve been leveraging with a lot of success. You can almost directly feed a package-lock.json into fetchTree which has an enormous performance impact compared to nixpkgs.fetch*.

3 Likes