Overriding python packages using overlays

Help
1

Hello all,

I’m new to both Nix and this forum, so please let me know if I need to modify
my post or clarify anything! I work as a software developer and we use Nix at
work. I’m currently tasked with upgrading our Nix channel (from 20.09 to
24.05), starting with Python, but I’ve encountered an issue while upgrading
from Python 3.6 to 3.12.

Specifically, we patch some of our Python dependencies and also have custom
Python modules. We use overlays to manage this setup, but I’m running into an
issue, and I’m still a bit confused about how overlays work, especially the
self and super parts.

Here’s a minimal example of my nix-shell configuration:

let
  nixpkgs-24-05 =
    (import (builtins.fetchTarball {
      name = "core-nixpkgs-pin-24-05";
      # latest of branch 24.05
      url = https://github.com/NixOS/nixpkgs/archive/944b2aea7f0a2d7c79f72468106bc5510cbf5101.tar.gz;
      sha256 = "sha256:0hrdi49yra4l1f7lm71zxd962m4fh365xq3c27k3di15c621arrn";
    }));

  pkgs = nixpkgs-24-05 {
    overlays = [
      (import ./overlay-python.nix)
    ];
  };
in
  pkgs.mkShell {
    packages = [
      (pkgs.python312.withPackages (python-pkgs: with python-pkgs; [
        pefile
      ]))
    ];
  }

This is my overlay file:

self: super: {
  python312 = super.python312.override {
    packageOverrides = self: super: {
      pefile = super.callPackage ./pkgs/pefile {};
    };
  };
}

However, when I try to run nix-shell, I get the following error:

error:
       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:34:12:
           33|
           34|   strict = derivationStrict drvAttrs;
             |            ^
           35|

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/xsm9npjlzjmm80h9d9qdw743lgmnkfy6-core-nixpkgs-pin-24-05/pkgs/stdenv/generic/make-derivation.nix:333:7

       … while evaluating attribute 'nativeBuildInputs' of derivation 'nix-shell'
         at /nix/store/xsm9npjlzjmm80h9d9qdw743lgmnkfy6-core-nixpkgs-pin-24-05/pkgs/stdenv/generic/make-derivation.nix:377:7:
          376|       depsBuildBuild              = elemAt (elemAt dependencies 0) 0;
          377|       nativeBuildInputs           = elemAt (elemAt dependencies 0) 1;
             |       ^
          378|       depsBuildTarget             = elemAt (elemAt dependencies 0) 2;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'callPackage' missing
       at /home/nsalaun/Projects/core-env/utils/distributions/nix/overlay-python.nix:4:16:
            3|     packageOverrides = self: super: {
            4|       pefile = super.callPackage ./pkgs/pefile {};
             |                ^
            5|     };

For reference, this is the version of Nix I’m using:

$ nix --version
nix (Nix) 2.24.9

I’m not sure why super doesn’t have the callPackage attribute, as it seems to
work fine with Python 3.6. I’ve tried dumping the contents of both self and
super, and they look quite similar, but super is missing callPackage, while
self has it. Should I be using super or self in this case, and why?

Apologies if my question is unclear — I’m still getting the hang of Nix and
overlays.

1 Like