Poetry2nix PyTables error: metadata-generation-failed

Help
1

Hello,

I am keen to hear any advice or thoughts on how to resolve the aforementioned error. I have included a fairly detailed description of how I reached this point.

cd discourse-poetry-tables
nix develop
poetry new baz # init python project
# move new python content to top dir
mv baz baz.1
mv baz.1/* .
rmdir baz.1

Create package function in ./baz/dev.py

import pandas

def hello():
	print('pandas', pandas.__version__)
	print('tables', tables.__version__)

Make a poetry script be adding to ./pyproject.toml

[tool.poetry.scripts]
hello = 'baz.dev:hello'

Add pandas dep to pyproject

poetry add pandas

The project structure is:

.
├── baz
│   ├── dev.py
│   └── __init__.py
├── flake.lock
├── flake.nix
├── poetry.lock
├── pyproject.toml
└── README.md

Run the script

poetry run hello
# ... add some libs to LD_LIBRARY_PATH for numpy in the shellHook 
pandas 1.5.3 # as desired

The pyproject.toml is

name = "baz"
version = "0.1.0"

[tool.poetry.dependencies]
python = "^3.10"
pandas = "^1.5.3"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.scripts]
hello = 'baz.dev:hello'

Now let’s use poetry2nix

nix build .#our-app
./result/bin/hello
pandas 1.5.3 # hurrah!

The flake.nix is

{
  description = "Application packaged using poetry2nix";

  inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  inputs.poetry2nix = {
    url = "github:nix-community/poetry2nix";
    inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, nixpkgs, flake-utils, poetry2nix }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        # see https://github.com/nix-community/poetry2nix/tree/master#api for more functions and examples.
        inherit (poetry2nix.legacyPackages.${system})
          mkPoetryApplication
          overrides
          defaultPoetryOverrides;

        pkgs = nixpkgs.legacyPackages.${system};

        our-app = mkPoetryApplication {
          projectDir = ./.;
        };
 
       in rec
      {
        packages = {
          our-app = our-app;
          default = self.packages.${system}.our-app;
        };

        devShells.default = poetry-dev;

        poetry-dev = pkgs.mkShell {
          packages = [
            poetry2nix.packages.${system}.poetry
            pkgs.hdf5.dev
            pkgs.zlib
          ];
          shellHook = ''
          export HDF5_DIR=${pkgs.hdf5.dev}/lib
          export LD_LIBRARY_PATH="${pkgs.lib.makeLibraryPath [ pkgs.zlib ]}:$LD_LIBRARY_PATH"
          export LD_LIBRARY_PATH="${pkgs.stdenv.cc.cc.lib.outPath}/lib:$LD_LIBRARY_PATH"
           '';
        };

      });
}

Now let’s add python tables PyTables. Update script

import pandas
import tables


def hello():
    print('pandas', pandas.__version__)
    print('tables', tables.__version__)

and add the deprmdency

poetry add tables

pyproject.toml:

...
[tool.poetry.dependencies]
python = "^3.10"
pandas = "^1.5.3"
tables = "^3.8.0"
...

and run

poetry run hello
pandas 1.5.3
tables 3.8.0 # good

Now poetry2nix

nix build .\#our-app

error: builder for '/nix/store/yj9fs95dn0i4w9q1pj09x3hk72p08kfl-python3.10-tables-3.8.0.drv' failed with exit code 1;
       last 10 log lines:
       >   full command: /nix/store/0pyymzxf7n0fzpaqnvwv92ab72v3jq8d-python3-3.10.9/bin/python3.10 /nix/store/wf1vywz8lb8xvh27zwcl41kc9hwl2r3p-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /build/tmpj5kwx84k
       >   cwd: /build/tables-3.8.0
       >   Preparing metadata (pyproject.toml) ... error
       > error: metadata-generation-failed
       >
       > × Encountered error while generating package metadata.
       > ╰─> See above for output.
       >
       > note: This is an issue with the package mentioned above, not pip.
       > hint: See above for details.
       For full logs, run 'nix log /nix/store/yj9fs95dn0i4w9q1pj09x3hk72p08kfl-python3.10-tables-3.8.0.drv'.
error: 1 dependencies of derivation '/nix/store/cllx9fc28ld65hcfkxm7932rlcfkvkpw-python3.10-baz-0.1.0.drv' failed to build

The nix log ... includes

      exec(code, locals())
    File "<string>", line 20, in <module>
  ModuleNotFoundError: No module named 'packaging'
  error: subprocess-exited-with-error

Many thanks

2

I have managed to move the table build along by updating the flake:

...
   our-app = mkPoetryApplication {
          projectDir = ./.;
          # overrides = defaultPoetryOverrides;
          overrides = overrides.withDefaults (self: super: {
            tables = super.tables.overridePythonAttrs (
              old: {
                nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [
                  self.packaging
                  self.blosc2
                ];                
              }
            );

            blosc2 = super.blosc2.overridePythonAttrs (
              old: {
                nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [
                  self.setuptools
                  self.scikit-build
                ];                
              }
            );
            
          });
        };
...

Which errors with

Building wheels for collected packages: blosc2
  Running command Building wheel for blosc2 (pyproject.toml)
  Traceback (most recent call last):
    File "/nix/store/b7vc72rhzj9adc39wk6pbmwrygmc2yh4-python3.10-scikit-build-0.16.4/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 612, in setup
      cmkr = cmaker.CMaker(cmake_executable)
    File "/nix/store/b7vc72rhzj9adc39wk6pbmwrygmc2yh4-python3.10-scikit-build-0.16.4/lib/python3.10/site-packages/skbuild/cmaker.py", line 149, in __init__
      self.cmake_version = get_cmake_version(self.cmake_executable)
    File "/nix/store/b7vc72rhzj9adc39wk6pbmwrygmc2yh4-python3.10-scikit-build-0.16.4/lib/python3.10/site-packages/skbuild/cmaker.py", line 104, in get_cmake_version
      raise SKBuildError(

  Problem with the CMake installation, aborting build. CMake executable is cmake