There are a bunch of packages that take a lockfile from $language_package_manager and make some derivations without generating any .nix files.
I wonder if it’s time to look at what they have in common and see what’s worth sharing.
What other ones are out there?
I think what’s in common is that the lock files of these languages/tools include the hash of the package to be downloaded and the URL of the package-files can be determined from the lock file.
I think the same thing can be done now with NPM7’s package-lock.json, as NPM7 now includes the URLs and hashes of all dependencies.
I haven’t used yarn2nix, but it looks like that generates a nix file? In this thread I’m mostly interested in ones that don’t generate any nix files.
Generating a nix file is optional. mkYarnPackage can directly use the lockfile to get get dependencies. You can actually just pass the source to mkYarnPackage and it will look for a yarn.lock and build the package with that.
I think most yarn packages in nixpkgs do generate the yarn.nix file and commit that. I’m not sure why, but I assume there’s a good reason for doing so.
It’s probably because of the cost of “import from derivation”, which would require to build something before beeing able to import a file from it. To import the freshly built file everything had to get re-evaluated. Roughly doubling the evaluation time.
Hydra strictly forbids IFD.
It doesn’t have to be import from derivation. If the lockfile is in json format, nix can import it as-is without an derivation that results in nix code.
This way it is possible to read the json file, convert the structure to one that fits a mkYarnPackage-like call. It probably maps urls and their hashes to fetchurl calls.
The same can be done with other text-based files (like Gemfile.lock) by reading the file as a string and using the Nix built-in functions to for instance split the string into lines and parse lines that way. The built-in functions are quite limited so I presume this isn’t the right way to go.
It would probably require more compute time to convert such files compared to a pregenerated nix file, but it is very different from an IFD. In addition, I thought Nix 3 would be doing more evaluation caching, so such computations will become faster in the future.
Yes, but the lockfile needs to exist within the nixpkgs repository.
Ah right, for nixpkgs that isn’t of much help. For project repos that want to add development shell or package the project using nix it seem seem to be ideal. In this case is doesn’t require IFD.