Poetry2nix and pycairo

Help
1

Hello! I’m in the final stretch of getting a development environment up and running with cairo, but mkPoetryEnv is failing when it tries to build pycairo with ninja. Any help or advice would be much appreciated.

It’s a wierd one for me, as the shell environment built, and I was able to poetry add pycairo and import it to ipython, but then when I exited the shell re-ran it later, it fails.

I based my shell.nix on the flake from this post here. My error looks like this:

source root is pycairo-1.24.0
setting SOURCE_DATE_EPOCH to timestamp 1687160849 of file pycairo-1.24.0/setup.cfg
patching sources
configuring
no configure script, doing nothing
building
build flags: -j4
ninja: error: loading 'build.ninja': No such file or directory
error: builder for '/nix/store/2kjdbsj2719yak5wxga0j1kq1z7kb2pr-python3.11-pycairo-1.24.0.drv' failed with exit code 1;
       last 10 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/v2kinyrcs3mh4jpwnq9z32727jjp43br-pycairo-1.24.0.tar.gz
       > source root is pycairo-1.24.0
       > setting SOURCE_DATE_EPOCH to timestamp 1687160849 of file pycairo-1.24.0/setup.cfg
       > patching sources
       > configuring
       > no configure script, doing nothing
       > building
       > build flags: -j4
       > ninja: error: loading 'build.ninja': No such file or directory
       For full logs, run 'nix log /nix/store/2kjdbsj2719yak5wxga0j1kq1z7kb2pr-python3.11-pycairo-1.24.0.drv'.
error: 1 dependencies of derivation '/nix/store/iz2j8g1bh7qvvx6jfq7bsrdh0jwcr9cc-python3-3.11.1-env.drv' failed to build

And my shell.nix so far is this:

{ pkgs ? import <nixpkgs> {} }:

let

  poetryEnv = pkgs.poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    preferWheels = true;
    python = pkgs.python311;
    overrides = pkgs.poetry2nix.overrides.withDefaults (self: super: {
      pycairo = super.pycairo.overridePythonAttrs (
        old: {
          nativeBuildInputs = [
            self.meson
            pkgs.buildPackages.ninja
            pkgs.buildPackages.pkg-config
            pkgs.ninja
            # pkgs.cairo
            # pkgs.gtk3
          ];
          propogatedBuildInputs = [ pkgs.cairo ];
        }
      );
      pygobject = super.pygobject.overridePythonAttrs (
        old: {
          buildInputs = (old.buildInputs or [ ]) ++ [ super.setuptools ];
        }
      );
      overrides = super.overrides.overridePythonAttrs (
        old: {
          buildInputs = (old.buildInputs or [ ]) ++ [ super.setuptools ];
        }
      );
      attrs = super.attrs.overridePythonAttrs (
        old: {
          buildInputs = (old.buildInputs or []) ++ [ super.hatchling ];
        });
    });
  };
  
in
  pkgs.mkShell {
    nativeBuildInputs = [
      pkgs.python311
      ( pkgs.poetry.override { python = pkgs.python311; } )
      pkgs.pandoc
      pkgs.cairo
      pkgs.pkg-config
      # pkgs.python311Packages.gst-python
      pkgs.gobject-introspection
      pkgs.exa
      pkgs.gtk3-x11
      pkgs.fd
      poetryEnv
    ];
    shellHook = '' 
      export XDG_DATA_DIRS=$GSETTINGS_SCHEMA_PATH
      SOURCE_DATE_EPOCH=$(date +%s)
      alias jn="jupyter notebook"
      alias ls=exa
      alias find=fd 
    '';
  }
2

I tried building ninja with certain flags, like below, but unfortunately, that tries to download ninja source from github in a derivation, and apparently there’s no network access allowed.

  python = pkgs.python3;
  ninja = python.pkgs.buildPythonPackage {
    pname = "ninja";
    version = "1.11.1";

    src = python.pkgs.fetchPypi {
      pname = "ninja";
      version = "1.11.1";
      # sha256 = "sha256-yDOkfTmy0e7j+cqIb6FYHv1b5gaLgnNKwimWHuh0j5A=";
      sha256 = "sha256-yDOkfTmy0e7j+cqIb6FYHv1b5gaLgnNKwimWHuh0j5A=";
    };

    nativeBuildInputs = [
      python.pkgs.scikit-build
      python.pkgs.setuptools_scm
      python.pkgs.setuptools
      pkgs.cmake
    ];
    format = "other";
    # dontUseCmakeConfigure = true;
    
  };
3

I’m facing the same problem, were you able to understand it?
I noticed that:

  1. a nix build nixpkgs#python311Packages.pycairo works without problems
  2. if I explicitly pin pycairo to 1.23.0 in the pyproject.toml, and not using 1.24.0 coming from nixpkgs, it gets built correctly
4

Hello! Sorry, I missed your message. No, I never solved this issue. It has something to do with this being a flake, and flakes needing to be pure and deterministic. Sorry I can’t be of more help. For what it’s worth, I started using Nixos again, and I’ll take another run at getting pycairo working. If I work something out, I’ll let you know.

5

I spent some time on this recently, and I found a workaround (or an hack, depending on how you view it :upside_down_face:).

I had to rewrite most of the poetry2nix’s pycairo expression adding an explicit copy of the shared library in the place where it is expected.

Maybe this rings the right bell to you…

Here is my pycairo entry in the poetry’s overrides:

  # For some reason, since a while the build of pycairo fails with an error
  # like
  #
  #   ERROR: File 'cairo/_cairo.cpython-311-x86_64-linux-gnu.so' could not be found
  #
  # because that file gets copied somewhere else. Here we explicitly copy it
  # under the build/cairo directory.
  #
  # See also https://discourse.nixos.org/t/poetry2nix-and-pycairo/30173
  pycairo = super.pycairo.overridePythonAttrs (old: {
    format = "other";
    nativeBuildInputs = old.nativeBuildInputs or [ ] ++ [
      self.setuptools
      self.meson
      pkgs.ninja
      pkgs.buildPackages.pkg-config
    ];
    propagatedBuildInputs = old.propagatedBuildInputs or [ ] ++ [
      pkgs.cairo
    ];
    preBuild = ''
      cd ../
    '';
    postBuild = ''
      cd build
      cp `find lib* -name '_cairo.*.so'` cairo
    '';
    mesonFlags = [ "-Dpython=${if self.isPy3k then "python3" else "python"}" ];
  });
1 Like
6

I had the exact same problem but the above solution didn’t work for me, I had to change it to

              pycairo = super.pycairo.overridePythonAttrs (old: {
                format = "other";
                nativeBuildInputs = old.nativeBuildInputs or [ ] ++ [
                  self.setuptools
                  self.meson
                  pkgs.ninja
                  pkgs.buildPackages.pkg-config
                ];
                propagatedBuildInputs = old.propagatedBuildInputs or [ ]
                  ++ [ pkgs.cairo ];
                preInstall = ''
                  cp `find lib* -name '_cairo.*.so'` cairo
                '';
                mesonFlags =
                  [ "-Dpython=${if self.isPy3k then "python3" else "python"}" ];
              });

So i changed postBuild to preinstall and removed the cd build. the prebuild hook wasn’t necessary for it to work so i removed that too.

No idea why though, I’m just hitting stuff with hammers until it works.

1 Like