Hi! I released the first version of pynixify, a project to automatically generate Nix expressions for Python projects. It is quite similar to @DavHau’s mach-nix in the way it detects dependencies, but it’s more focused on intermediate Nix users. Some features provided by pynixify are:
Generates human-readable expressions that follow nixpkgs styling conventions: The output of the command can be copied to nixpkgs/pkgs/development/python-modules/. After a few manual changes, you’ll be able to submit a PR to nixpkgs adding new Python packages.
Reuses the expressions available in nixpkgs instead of creating them from scratch, making the expressions more reliable
Supports nixpkgs pinning to enforce reproducibility between different machines
It’s Free Software, licensed under the GPLv3 license
Looks nice!
I have been playing with the thought of adding an upstreamable output format to mach-nix for some time now, but never realized it.
Could you elaborate what are the differences between pynixify and python-package-init from nixpkgs-pytools?
Should we create an overview in the wiki, listing the different python helper tools and their capabilities? It might save some people some time since there are around a handful of tools by now.
nixpkgs-pytools: This library also targets creating packages for nixpkgs. Unlike pynixify, it doesn’t automatically create expressions for the missing dependencies, and the generated expressions require manual tweaking before being valid Nix. pynixify would be located in the middle between nixpkgs-pytools and mach-nix.
Could you elaborate what are the differences between pynixify and python-package-init from nixpkgs-pytools?
nixpkgs-pytools: This library also targets creating packages for nixpkgs. Unlike pynixify, it doesn’t automatically create expressions for the missing dependencies, and the generated expressions require manual tweaking before being valid Nix. pynixify would be located in the middle between nixpkgs-pytools and mach-nix.
There is the summary of their differences, but I’ll ellaborate a bit more based on my experience using nixpkgs-pytools yersterday (it was the first time I heard of it).
The expression generated by pytools is intended to be a template. It isn’t even syntactically valid without manual interaction:
meta = with lib; {
description = "Collaborative Penetration Test and Vulnerability Management Platform https://www.faradaysec.com";
homepage = https://github.com/infobyte/faraday;
license = licenses.;
# maintainers = [ maintainers. ];
};
From the point of view of pytools, this makes sense. The tool is intended to be used only by Nixpkgs maintainers to avoid writing boilerplate code.
My usecase for writing pynixify is somewhat different: I work as a developer of an open-source product with lots of Python dependencies. Adding it to nixpkgs was one my objectives, but not the only one. I also wanted to make Nix usable in our product by (some of) my coworkers and to support our CI pipeline. This replaced the buggy apt install && pip install -r requirements.txt process.
So pynixify tries to target both problems:
First, it should create human-readable expressions, but they have to work out of the box. This implies having to get the proper attribute names for each PyPI package in nixpkgs (done here), parsing the package dependencies in a way similar to mach-nix, and creating expressions for the unpackaged dependencies. All of these have to be done manually with pytools, but pynixify does it automatically. I don’t think one tool is better than another. They’re just different approaches.
Also, it needs to be usable in work environments where not everyone is familiarized with Nix. Adding a dependency to the package’s requirements shouldn’t require writing lots of Nix code. It’d be better to automatically generate it. This usecase is similar to mach-nix’s. Actually, I started writing pynixify before mach-nix was released. Otherwise, I would have considered contributing to it instead.
I hope this gave a broader background on the tool’s objectives.
python2nix has been unmaintained for 3 years and pypi2nix never really worked for me. Based on this recent blogpost, poetry2nix maintainers feel the same way about pypi2nix.