Announcing pynixify: Nix expression generator for Python projects

Hi! I released the first version of pynixify, a project to automatically generate Nix expressions for Python projects. It is quite similar to @DavHau’s mach-nix in the way it detects dependencies, but it’s more focused on intermediate Nix users. Some features provided by pynixify are:

  • Generates human-readable expressions that follow nixpkgs styling conventions: The output of the command can be copied to nixpkgs/pkgs/development/python-modules/. After a few manual changes, you’ll be able to submit a PR to nixpkgs adding new Python packages.
  • Reuses the expressions available in nixpkgs instead of creating them from scratch, making the expressions more reliable
  • Supports nixpkgs pinning to enforce reproducibility between different machines
  • It’s Free Software, licensed under the GPLv3 license

I’ve been using pynixify to introduce Nix at my job with a great success, and to contribute with some PRs to nixpkgs in an easy way.

You can find the project in its GitHub page: https://github.com/cript0nauta/pynixify. Contributions, feature requests, and bug reports are more than welcome!

3 Likes

Looks nice!
I have been playing with the thought of adding an upstreamable output format to mach-nix for some time now, but never realized it.

Could you elaborate what are the differences between pynixify and python-package-init from nixpkgs-pytools?

Should we create an overview in the wiki, listing the different python helper tools and their capabilities? It might save some people some time since there are around a handful of tools by now.

1 Like

There is a list here: https://nixos.wiki/wiki/Language-specific_package_helpers
But no in-depth comparison is made.

In the readme it says:

  • nixpkgs-pytools: This library also targets creating packages for nixpkgs. Unlike pynixify, it doesn’t automatically create expressions for the missing dependencies, and the generated expressions require manual tweaking before being valid Nix. pynixify would be located in the middle between nixpkgs-pytools and mach-nix.

Could you elaborate what are the differences between pynixify and python-package-init from nixpkgs-pytools?

  • nixpkgs-pytools: This library also targets creating packages for nixpkgs. Unlike pynixify, it doesn’t automatically create expressions for the missing dependencies, and the generated expressions require manual tweaking before being valid Nix. pynixify would be located in the middle between nixpkgs-pytools and mach-nix.

There is the summary of their differences, but I’ll ellaborate a bit more based on my experience using nixpkgs-pytools yersterday (it was the first time I heard of it).

The expression generated by pytools is intended to be a template. It isn’t even syntactically valid without manual interaction:

  meta = with lib; {
    description = "Collaborative Penetration Test and Vulnerability Management Platform https://www.faradaysec.com";
    homepage = https://github.com/infobyte/faraday;
    license = licenses.;
    # maintainers = [ maintainers. ];
  };

From the point of view of pytools, this makes sense. The tool is intended to be used only by Nixpkgs maintainers to avoid writing boilerplate code.

My usecase for writing pynixify is somewhat different: I work as a developer of an open-source product with lots of Python dependencies. Adding it to nixpkgs was one my objectives, but not the only one. I also wanted to make Nix usable in our product by (some of) my coworkers and to support our CI pipeline. This replaced the buggy apt install && pip install -r requirements.txt process.

So pynixify tries to target both problems:

  • First, it should create human-readable expressions, but they have to work out of the box. This implies having to get the proper attribute names for each PyPI package in nixpkgs (done here), parsing the package dependencies in a way similar to mach-nix, and creating expressions for the unpackaged dependencies. All of these have to be done manually with pytools, but pynixify does it automatically. I don’t think one tool is better than another. They’re just different approaches.
  • Also, it needs to be usable in work environments where not everyone is familiarized with Nix. Adding a dependency to the package’s requirements shouldn’t require writing lots of Nix code. It’d be better to automatically generate it. This usecase is similar to mach-nix’s. Actually, I started writing pynixify before mach-nix was released. Otherwise, I would have considered contributing to it instead.

I hope this gave a broader background on the tool’s objectives.

I think we should update not only the wiki but the nixpkgs manual too. Here is a fragment of it:

15.17.2.4. Tools

Packages inside nixpkgs are written by hand. However many tools exist in community to help save time. No tool is preferred at the moment.

python2nix has been unmaintained for 3 years and pypi2nix never really worked for me. Based on this recent blogpost, poetry2nix maintainers feel the same way about pypi2nix.

1 Like