Add Python package via overlay

Help
1

I know how to use overlays to add local packages to nixpkgs using this sort of approach:

pkgs = import nixpkgs {
  overlays = [
    (final: prev: {
      my-package = final.callPackage ./overlays/my-package.nix {};
    })    
  ];
};

However, this does not work if the package is a Python package, as those need to go one level deeper: inside python<N>Packages.

I have found instructions in the official manual showing how to override parts of an existing Python package, but for the case of adding a new Python package, I haven’t managed to find an approach that works.

Can you suggest a reliable recipe for adding a Python package using overlays?

(I’m trying to do this in a flake, in case that changes anything.)

2

I typically use what I documented in pythonPackagesExtensions: override all Python package sets at once by FRidh · Pull Request #91850 · NixOS/nixpkgs · GitHub.

3

The PR documents a pattern like this

(final: prev: {
  pythonPackagesOverlays = prev.pythonPackagesOverlays ++ [
    (
      python-final: python-prev: {
        foo = ...;
      }
    )
  ];
})

but that same commit introduces numerous changes to various files in the nixpkgs infrastructure, from which I infer that the documented approach depends on those changes. As the PR has not been merged, it would seem that this approach is not going to work in a current nixpkgs. What have I misunderstood?

4

A self-contained version could look like the one below:

final: prev:
{
  pythonPackagesOverlays = (prev.pythonPackagesOverlays or [ ]) ++ [
    (python-final: python-prev: {
      # cviz = python-final.callPackage cvizPkg { };
      # ...
    })
  ];

  python3 =
    let
      self = prev.python3.override {
        inherit self;
        packageOverrides = prev.lib.composeManyExtensions final.pythonPackagesOverlays;
      }; in
    self;

  python3Packages = final.python3.pkgs;

}

I’m using exactly this snippet both in configuration.nix and in project environments

EDIT 2023-08-14: Since a while ago you can skip python3 = let self = ... ; in self and use pythonPackagesOverrides instead: Nixpkgs 24.05 manual | Nix & NixOS

1 Like
5

Thanks! I can get this to work for a single python version.

Any ideas how it could be adapted to work with multiple versions? Specifically, can I avoid hard-wiring like in this commit:

https://github.com/jacg/nix-starters/commit/aed6a83780e02a3b1fecfa5cef541fb2b7b407d7

?

6

You could define a single packageOverrides and apply it to several attributes (python39, python310, etc). Similarly, you can define the filter ps: [ ... ] once, and use it for multiple python wrappers. Are you sure you really need any of this though?

7

You’re probably right, it’s not worth the effort.

The only practical use I’ve had for this kind of thing is to have CI notice when new Python versions become usable with all my dependencies.