NUR repo recursive python packages

Help
1

I really like testing new packages in my own nur repo before i submit them to nixpkgs, but a lot of them require a bunch of python modules and I’m not quite sure how to best add those.

I wanted to have my own directory for them like nixpkgs

pkgs
│   ├── development
│   │   └── python-modules
│   │       ├── baron
│   │       │   └── default.nix
│   │       ├── redbaron
│   │       │   └── default.nix
│   │       └── default.nix
│   ├── PackageThatRequiresNewPythonModules
│   │   └── default.nix

My current approach is having pkgs/development/python-modules/default.nix declare all py modules:

{ callPackage }:

rec {
  baron = callPackage ./baron { };

  redbaron = callPackage ./redbaron { }
}

and then in my main top-level default.nix:

{
...
  python3Packages = pkgs.python3Packages // pkgs.recurseIntoAttrs (
    (pkgs.python3Packages.callPackage ./pkgs/development/python-modules {
    })
  );

  PackageThatRequiresNewPythonModules = pkgs.callPackage ./pkgs/PackageThatRequiresNewPythonModules { inherit python3Packages; };
}

Which except for having to write inherit python3Packages; everytime works pretty well, but now I have a new python module that itself depends on a python module not in nixpkgs (here redbaron requires baron) and I don’t know how to make these aware of each other.

2

what you’re looking for is makeScope.

example usage:

let
  myPackages = lib.makeScope python3Packages.newScope (self: with self; {
     packageA = callPackage ./mypackageA.nix { };
     ....
  });
in myPackages.callPackage ./package.nix { };

Anything not found at the myPackages scope, will then be “brought in” from the python3Packages scope, if it’s not in python3Packages, then it will be brought in from the pkgs scope. This will allow you create package expressions which feel like you’re extending the python3Packages package set (well… because you are :slight_smile: ). In package.nix, you can refer to packageA and it will be able to find it.

3 Likes
3

Thank you that makes a lot of sense, but with especially python3Packages I always just get:

attribute 'newScope' missing,

I tried to replicate what I saw with xfce but using pkgs.python3Packages instead

Using pkgs.xfce.newScope works, obviously then the callPackage will be wrong. Is the python packages scope called different?

4

python3.pkgs or pkgs.python3Packages they are aliases

5

right, but I can’t seem to be able to use newScope on either.

error: while evaluating 'newScope' at /nix/store/nl8kcg273n8f852p8z6yshrzin7ika3j-nixos-20.09.4064.1b688ca59ba/nixos/lib/customisation.nix:210:22, called from /nix/store/nl8kcg273n8f852p8z6yshrzin7ika3j-nixos-20.09.4064.1b688ca59ba/nixos/lib/customisation.nix:211:25:
while evaluating the attribute 'pkgs.newScope' at /nix/store/nl8kcg273n8f852p8z6yshrzin7ika3j-nixos-20.09.4064.1b688ca59ba/nixos/pkgs/development/interpreters/python/default.nix:38:9:
attribute 'newScope' missing, at /home/kaine/git/nur-packages/pkgs/development/python-modules/default.nix:3:15

{ config, lib, python3 }:

lib.makeScope python3.pkgs.newScope (self: with self; {
    baron = callPackage ./baron { };
    redbaron = callPackage ./redbaron { };
})
6

Looking through the source, it appears pythonpackage doesn’t implement itself using the the newScope helper, so it also doesn’t export it. :frowning:

You’re probably limited to doing packageOverrides then… ;(

1 Like
7

aww! guess I’ll go with that, but newScope has already become useful in different contexts, so thanks!