How to handle packaging python packages that pin dependency version

I am packaging a python package and the developer decided to pin a dependency to an exact version. How should this be handled? Here is an example. This package seems to ignore me explicitly providing a different version as a dependency.

Is this related to Python's packageOverrides isn't composable · Issue #44426 · NixOS/nixpkgs · GitHub ?

{ stdenv
, buildPythonPackage
, fetchPypi
, pyyaml
, requests
, coverage
, six
, pytest
, pytestpep8
, pytestcov
, httpretty
}:

let
   # requires exact version of coverage 4.0.3
   coverage-403 = coverage.overrideDerivation (oldAttrs: rec {
      version = "4.0.3";
      src = fetchPypi {
         pname = oldAttrs.pname;
         inherit version;
         sha256 = "85b1275b6d7a61ccc8024a4e9a4c9e896394776edce1a5d075ec116f91925462";
      };
    });
in
buildPythonPackage rec {
  version = "2.9.1";
  pname = "python-coveralls";

  src = fetchPypi {
    inherit pname version;
    sha256 = "736dda01f64beda240e1500d5f264b969495b05fcb325c7c0eb7ebbfd1210b70";
  };

  checkInputs = [ pytest pytestpep8 pytestcov httpretty ];
  propagatedBuildInputs = [ pyyaml requests coverage-403 six ];

  checkPhase = ''
    py.test coveralls/tests.py
  '';

  meta = with stdenv.lib; {
    homepage = http://github.com/z4r/python-coveralls;
    description = "Python interface to coveralls.io API";
    license = licenses.asl20;
    maintainers = [ maintainers.costrouc ];
  };
}

Or would it be better to just use sed and replace the dependency and allow newer versions? (Obviously there might be a reason the developer pinned the package in the first place)

1 Like

This is a great example: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/home-assistant/default.nix

2 Likes

Exactly what I was looking for. Thanks!

Before you go ahead and do that, the first thing I’d check is to make sure that it actually needs the pin. Often, developers pin their packages unnecessarily – particularly if it’s an application and they just want to make sure it works for end users.

If the pins aren’t needed, you can get smaller, faster, more efficient closures by just removing them, since you won’t have to download and install multiple versions of libraries. This is also less likely to result in diamond dependency problems or inconsistent versions. There are many examples of this pattern in python packages; here are two:

1 Like

@bhipple I get what you’re saying, but is that really the way to go? You can’t expect software to work correctly on any version of its dependencies. The only way to ensure for future upgrades this is to run the automated tests of the software, is that done for all python packages?

Yes in contrast to “regular” nix packages, python packages default to doCheck = true (assuming there are tests to be run).

I think the better way to do this is to make a new attribute in the python package set coverage_4_0_3 and then pass {coverage = coverage_4_0_3; } when calling the package which needs it?

We aim at having only a single version of each package to prevent collisions. If it works, unpin it. If not, and the package is an application, then provide another private version of the dependency. If it’s a library and you want to have it in Nixpkgs, well, too bad.

Also, you’ll typically want overridePythonAttrs instead of overrideAttrs or overrideDerivation.