A procedure for working with mach-nix and misbehaving Python packages

I often create nix environments/derivations/flakes with other people’s Python packages using mach-nix. Very frequently mach-nix won’t build the Python packages due to "Requirements conflict"s. Sometimes overriding the requirements in mach-nix suffices (using the mach-nix requirements attribute) but if I need to remove a dependency the requirements of the Python package must be changed. I find the following procedure to be most convenient:

  • Copy the Python packages requirements.txt the nix project dir
  • Create a patch that removes the requirements.txt from the Python package (sometimes I also edit setup.py)
  • Edit the copy of requirements until a satisfactory build is achieved

More specifically:

Starting from my Nix project repo

git clone some/.../baz.git
cd baz
cp requirement.txt ../requirements-baz.txt
rm requirements.txt
git diff > ../baz.patch
cd ..
rm -rf baz

Now I use a block like this in my derivation

baz = machNix.buildPythonPackage {
	src = builtins.fetchGit{
		url = "https://.../baz.git";
		ref = "master";
		rev = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
        };
    patches = [
      ./baz.patch # deletes package requirements.txt
    ];
    requirements = builtins.readFile ./requirements-baz.txt ;
};

Here I edit requirements-baz.txt.

Perhaps someone will find this useful.

2 Likes

Do pin your pypi database? Each release of mach-nix pins the pypi-database to a specific commit and this gets in the way of the package resolution when the release is getting older

1 Like

That’s a good point, as here.

I have yet to successfully resolve my build issues by updating the pypi database, but good to remember.

Would you happen to know how to specify the pypi database in a flake?

1 Like

yes, it’s easy enough! i do this in my flakes:

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-21.11";
    pypi-deps-db = {
      url = "github:DavHau/pypi-deps-db";
      flake = false;
    };
    mach-nix = {
      url = "github:DavHau/mach-nix/3.5.0";
      inputs.pypi-deps-db.follows = "pypi-deps-db";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
...

This way, the pypi-deps-db commit is pinned using my flake’s lockfile and it’s injected in the mach-nix flake.

3 Likes