How do I use a specific patch release of Python (eg 3.7.5)?

Learn
#1

One of the main reasons I use Nix is to develop projects that mix tools and libraries in different languages, while being able to test with different versions of each. For Python, Nix makes it easy to try different minor versions: 3.6, 3.7, 3.8, 3.9.

However, some of my Python code will run in environments that don’t use Nix and have specific patch releases of Python: some work code runs with 3.7.1, other projects need 3.7.3 or 3.7.5… etc. Nixpkgs has python37 point to some patch (3.7.9 at the moment), with no clear way to change that. What’s the most idiomatic way to define versions of pkgs.python at specific patch releases. I’d like to have my own pkgs.python371 or pkgs.python375 to run Nix shells and tests in different environments.

I managed to hack something together for 3.7.1 specifically by defining an overlay, but it’s a bit messy and took a bunch of trial and error (figuring out the self argument was tricky!):

python-overlay = final: current:
  let
    python371 = current.python37.overrideAttrs (old: old // rec {
      name = "python-${version}";
      version = "3.7.1";
      src = final.fetchurl {
        url = "https://www.python.org/ftp/python/3.7.1/Python-3.7.1.tar.xz";
        sha256 = "0v9x4h22rh5cwpsq1mwpdi3c9lc9820lzp2nmn9g20llij72nzps";
      };
      patches = with current; with current.lib;
        optionals stdenv.isDarwin [
          (fetchpatch {
            url = https://bugs.python.org/file47666/darwin-libutil.patch;
            sha256 = "0242gihnw3wfskl4fydp2xanpl8k5q7fj4dp7dbbqf46a4iwdzpa";
          })
        ];
    });
  in
  {
    python371 = python371.override { self = python371; };
  };

What is the idiomatic way to do things like this with Nixpkgs?

#2

There shouldn’t be much difference from a usage standpoint between the patch versions. You normally shouldn’t see difference between 3.7.1 vs 3.7.3 vs 3.7.5.

Since nixpkgs also exports a large 3,000+ package set with each interpreter version, we try to limit it to the most relevant interpreters, which is usually the latest of each minor version.

Using an overlay, similar to the manner in which you are already doing it, would be the correct path, if you did want to pin to a patch version.

#3

There shouldn’t be much difference from a usage standpoint between the patch versions. You normally shouldn’t see difference between 3.7.1 vs 3.7.3 vs 3.7.5 .

Yep. But I’ve already had differences between these bite me :(.

Using an overlay, similar to the manner in which you are already doing it, would be the correct path, if you did want to pin to a patch version.

Okay, that makes sense. I understand why Nixpkgs would not have different patch versions directly.

I was hoping there was an easier/cleaner way to define the overlay—I remember that getting the 3.7.1 overlay working took a couple of days to debug and was pretty frustrating, and the code seems too coupled to implementation details in Nixpkgs. I only got it working after reading through a bunch of Nixpkgs code.

Sounds like adapting my old 3.7.1 overlay for my current project is the best option for now.

Thanks!

#4

Yes, patches sometime differ between patch releases. In that case you may better just copy the whole expression so you maintain it yourself easier.