Python Package Rodeo

Announcements
5

The root causes of mach-nix failing on cartopy are:

  1. nixpkgs’ sphinx package is missing certifi as a dependency. Nixpkgs maintainers are often not aware of misisng deps, since some sub dependency has the missing dependency inside checkInputs, making the dependency present for the hydra build. But as soon as doCheck is disabled, those dependencies are missing, and the build fails.
  2. cartopy doesn’t specify it’s dependencies on pypi in a way that is recognized by the crawler maintining mach-nix’ dependency DB. Usually mach-nix would automatically fix the missing dependencies, but it cannot in this case since there is no data about it.

This problem is easily fixable, by manually adding some of cartopy’s dependencies to the requirements to make mach-nix aware of their sub dependencies.

More specifically, since building sphinx and fiona fail with a missing dependency error, we just need to add sphinx and fiona to the requirements manually. By doing this, we make mach-nix aware those packages and their sub-dependencies, and therefore allow it to build a better nix expression containing all necessary dependencies instead of trusting nixpkgs.

I extended your expression a bit. This one should work:

let
  mach-nix = import (builtins.fetchGit {
    url = "https://github.com/DavHau/mach-nix/";
    ref = "refs/tags/3.2.0";
  }) {};

in
mach-nix.mkPython {
  requirements = ''
    cartopy
    tensorflow>=2
    docrep

    # manually add some of cartopy's deps to make mach-nix aware of their sub-deps
    sphinx == 3.1.1
    fiona == 1.8.18

    # add scipy to speed up build since mach-nix will use the wheel now
    scipy
  '';
}

Take away:
It would be nice if we could permanently fix the checkInputs dependency leakage in nixpkgs.

2 Likes