Recently I talked with Stéphane Bidoul in person. He is a core pip developer. I asked him about one problem I see with Python packaging when using nix: if I use a package whose run-time dependency has changed, I have to rebuild it always.
Turns out this only happens for nix. In the normal python world, you only need build-time dependencies to build a python package. Often, these are just flit
, poetry-core
, setuptools
, wheel
or similar. The package output should always be the same. Later, when you actually want to use it, is when you really need the run-time dependencies.
I can see that buildPythonPackage
can make use of those in buildInputs
, whereas it still requires us to add runtime dependencies as propagatedBuildInputs
. Those make sense as the build will perform some tests such as python-importing the module, or even running pytests or unittests of the package. However, even if those tests are useful, shouldn’t the build output still be the same?
So, could we do better? If the only things that impact in the build output are the buildInputs
, couldn’t we split the inner mechanism of buildPythonPackage
into 2 derivations? One that builds the package, and another one that just tests it and returns the previous one on success.
Couldn’t we replace the propagatedBuildInputs
list of derivations by a list of strings that, when evaluated, are obtained from python3Packages
or similar, and then produce the dependency chain depending on that input?
If we manage to do something like this, the amount of rebuilds across nixpkgs python packages will be reduced dramatically (I guess).