buildPythonPackage PEP518 PEP517

Help
1

Hi guys, I need to add my PyPI package to the system packages.
I built it with poetry it doesn’t have setup.py only pyproject.toml

I tried to test build but it fails

# default.nix
with import <nixpkgs> {};
with python39Packages;

buildPythonPackage rec {
  pname = "base16_colorlib";
  format = "pyproject";
  version = "0.2.0";
  src = fetchPypi {
    inherit pname version;
    sha256 = "f0e0eeb50e8f9af1a00950577f6178febcf80ab2bf9bad937f9fe8068936432c";
  };
  doCheck = false;
  BuildInputs = [ poetry ];   # or poetry-core doesn't matter
}

I can’t figure out how to tell the builder the correct build inputs

nix-build

If I specify poetry

error: poetry was promoted to a top-level attribute, use poetry-core to build Python packages
(use '--show-trace' to show detailed location information)

If I specify poetry-core

error: builder for '/nix/store/a3dl57jqvz2zwbl10rx0lgg177gknjyq-python3.9-base16_colorlib-0.2.0.drv' failed with exit code 2;
       last 10 log lines:
       >   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
       >   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
       >   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
       > ModuleNotFoundError: No module named 'poetry'

I’m not much confused, and I seem to lack knowledge (

I tried to find the right solution but I haven’t found it yet
There are two issue:
one
two

but they do not have examples suitable for my case, as well as in the manual

2

This only makes sense for CLI packages, not for individual Python modules.
Installing them as a part of your system packages will have no usable effect.

What I believe you actually want is a globally installed Python environment that you can use interactively?
If so what you’ll want to add is something using withPackages into your systemPackages like so:

environment.systemPackages = [ (python3.withPackages(ps: [ ps.requests ])) ];

Overriding the nixpkgs Python set requires quite a bit of care to do it correctly, a more complete example to achieve this is:

  python = pkgs.python39.override {
    self = python;
    packageOverrides = self: super: {

      base16_colorlib = self.buildPythonPackage rec {
        pname = "base16_colorlib";
        format = "pyproject";
        version = "0.2.0";
        src = self.fetchPypi {
          inherit pname version;
          sha256 = "f0e0eeb50e8f9af1a00950577f6178febcf80ab2bf9bad937f9fe8068936432c";
        };
        doCheck = false;
        nativeBuildInputs = [ self.poetry-core ];
      };

    };
  };

This creates an overriden Python interpreter along with a package set which contains our custom package.
Note that we have moved poetry-core to nativeBuildInputs.

To incorporate this into your NixOS config would look something like:

{ pkgs, config, ... }:

let
  # Create an overriden python that has our custom package
  python = pkgs.python39.override {
    self = python;
    packageOverrides = self: super: {

      base16_colorlib = self.buildPythonPackage rec {
        pname = "base16_colorlib";
        format = "pyproject";
        version = "0.2.0";
        src = self.fetchPypi {
          inherit pname version;
          sha256 = "f0e0eeb50e8f9af1a00950577f6178febcf80ab2bf9bad937f9fe8068936432c";
        };
        doCheck = false;
        nativeBuildInputs = [ self.poetry-core ];
      };

    };
  };

in
{
  environment.systemPackages = [
    (python.withPackages(ps: [
      ps.base16_colorlib
    ]))
  ];
}
1 Like
3

Yes, thank you very much, it really worked! :handshake:

Here is part of my home-manager configuration:

{
  config,
  pkgs,
  ...
}: 
let
  # Create an overriden python that has our custom package
  python = pkgs.python311.override {
    self = python;
    packageOverrides = self: super: {
      base16_colorlib = self.buildPythonPackage rec {
        pname = "base16_colorlib";
        format = "pyproject";
        version = "0.2.0";
        src = self.fetchPypi {
          inherit pname version;
          sha256 = "f0e0eeb50e8f9af1a00950577f6178febcf80ab2bf9bad937f9fe8068936432c";
        };
        doCheck = false;
        nativeBuildInputs = [ self.poetry-core ];
      };
    };
  };
in
{
  home.packages = with pkgs; [
    #.......
    (python.withPackages(ps: [ ps.base16_colorlib ]))
    #....

Here is the result, globally python has access to the package:

which python
/home/stepan/.nix-profile/bin/python
 python
Python 3.11.2 (main, Feb  7 2023, 13:52:42) [GCC 12.2.0] on linux
>>> from pprint import pprint
>>> from base16_colorlib import onedark
>>> pprint(onedark)
{'author': 'https://github.com/one-dark',
# ...
 'scheme': 'onedark'}
>>>

I will leave additional information in the hope that it can be useful to someone.
In fact, I tried to implement something similar in the configuration, only then I decided to try nix-build.

I did it like in the wiki:

{
  config,
  pkgs,
  ...
}: 
  let
    my-python-packages = ps: with ps; [
      # ...
      (
        buildPythonPackage rec {
          pname = "base16_colorlib";
          format = "pyproject";
          version = "0.2.0";
          src = fetchPypi {
            inherit pname version;
            sha256 = "f0e0eeb50e8f9af1a00950577f6178febcf80ab2bf9bad937f9fe8068936432c";
          };
          doCheck = false;
        BuildInputs = [ pkgs.python311Packages.poetry-core ];
        }
      )
    ];
  in
{
  home.packages = with pkgs; [
    #....
    (python311.withPackages my-python-packages)
   #...

It didn’t work. The error was the same as in my first post.
But it worked with packages that are in nixpkgs

#...
home.packages = with pkgs; [
    (python311.withPackages(ps: with ps; [ requests ]))
];

I think these lines solves the problem:

python = pkgs.python311.override
self = python;
nativeBuildInputs = [ self.poetry-core ]; 
packageOverrides...

It is necessary to study the work of the packageOverrides function)))
In any case, thank you very much, I always try to solve the problem myself, but this is probably not the case) you saved me a lot of time and nerves)