Python packaging with Poetry and Nix

Help
1

Hello all,

I am working on a python app using Poetry which has a pyproject.toml which looks like this:
(truncated in favour of brevity)

[tool.poetry.dependencies]
python = "^3.9"
click = "^8.1.3"
cookiecutter = "^2.1.1"
jinja2 = "^3.1.2"
pyyaml = "^6.0"
flatdict = "^4.0.1"
importlib-metadata = "^6.1.0"

[tool.poetry.scripts]
wflow = "workflows.cli:cli"

[tool.poetry.group.dev.dependencies]
isort = "^5.12.0"
pytest = "^7.2.2"
flit-core = "^3.8.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Following is a barebones flake file I am using:

{
  description = "Testing nix for python";

  inputs.nixpkgs.url = github:NixOS/nixpkgs?ref=23.05-pre;

  outputs = {self, nixpkgs} :
  let
    system = "x86_64-linux";
    pkgs = nixpkgs.legacyPackages.${system};
    pkg_prefix = "blah";
    pkg_version = "0.1.0";
    poetry_customOverrides = final: prev: {
      # Custom overrides
    };
  in
  {
    packages = {
      ${system}.default = pkgs.poetry2nix.mkPoetryApplication {
        projectDir = self;
        overrides = [pkgs.poetry2nix.defaultPoetryOverrides poetry_customOverrides];
      };
    };
  };
}

Now when I use poetry2nix to package the app, it throws an error which looks like:

root@c2397b986a3b:/build# nix build
error: builder for '/nix/store/hyy1hjkwsy5iw41mfmp7phchqvv0v3q2-python3.10-packaging-23.0.drv' failed with exit code 2;
       last 10 log lines:
       >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
       >   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
       >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
       > ModuleNotFoundError: No module named 'flit_core'

The issue is caused because of black package which is added as a dev dependency in Poetry. If I remove it, the error goes away. (I would like to learn how I can fix this error in a nix recommended manner instead of removing packages)

So I have 3 questions.

  1. How can we override the dependencies in a way to ignore(remove) black when building nix package?
  2. How can we inject the flit-core package into the nix package building process?
  3. Is there a way to create 2 packages using poetry2nix, one will be just the application and other will be purely the documentation for the application?

Thanks,
Bhavesh.

2

At least for 2. you could check the edge cases

2 Likes
3

Thanks @foolnotion. That did work out for me.

Do you know if we can also create a separate package for docs as well when using poetry2nix?
Worst case scenario, I guess we could just make a normal nix package using mkDerivation for docs.

Another question I have is related to the entrypoints in the python package.
Is there anyway that poetry2nix is able to inspect the pyproject.toml file and look at the tool.poetry.scripts and create an application for each of the specified entrypoints?

I am able to get the values myself, by using something like:

builtins.attrNames(builtins.fromTOML(builtins.readFile ./pyproject.toml));

But how can I run this into a for loop so I can have a function to create an app for each entrypoint listed in the pyproject.toml file?

Ideally something like (Pseudo code):

tool_names = builtins.attrNames(builtins.fromTOML(builtins.readFile ./pyproject.toml));

for tool_name in tool_names:
    apps.${system}.${tool_name} = {
          type = "app";
          program = "${self.packages.${system}.default}/bin/${tool_name}";
    };

Any guidance would be really appreciated!