Using a dependency built via poetry2nix, in a package that builds uses poetry2nix

Help
1

Hello all,

I have a nix flake, flakeA, which provides a python package, packageA, built using poetry2nix. I then have another flake, flakeB, which uses poetry2nix to build another python package, packageB. What is the best way to make packageA a dependency of packageB?

At the moment I’m doing this:

packageB = mkPoetryApplication {
    projectDir = self;
    propagatedBuildInputs = [ packageA ];
  };

This causes errors like this:

       > pythonCatchConflictsPhase
       > Found duplicated packages in closure for dependency 'pytest':
       >   pytest 7.4.0 (/nix/store/0n34056sv9vnsz72mp5pf2d3ya41dkpy-python3.10-pytest-7.4.0/lib/python3.10/site-packages)
       >   pytest 7.4.0 (/nix/store/ilcsw0aq8crp9xigya7mbcnz6m84qbqs-python3.10-pytest-7.4.0/lib/python3.10/site-packages)

So it seems that packageA is trying to bring its library dependencies with it. Is there a better way to do this?

Thanks!

2

Have a look at https://github.com/nix-community/poetry2nix/blob/1d7eda9336f336392d24e9602be5cb9be7ae405c/docs/edgecases.md
You need to use an override on the buildInputs of packageA to filter out pytest (it should go into checkInputs).

3

Filtering is not documented in the above linked file. Would you mind posting an example how to do it?

4

I haven’t yet had the chance to try the suggestion above (@wamserma, thank you very much for that) but I think I conceptually understand it. If I can make it work, I’ll post what I did here.

5

@supermarin With an overlay like

{
  packageA = super.packageA.overridePythonAttrs (
    old: {
      nativeBuildInputs = lib.filter (x: x.pname != "pyest") old.nativeBuildInputs;
    }
  );
}

or maybe an override is enough:

let 
  packageA' = packageA.overridePythonAttrs (
    old: {
      nativeBuildInputs = lib.filter (x: x.pname != "pyest") old.nativeBuildInputs;
    }
  );
in
  # ...
  propagatedBuildInputs = [ packageA' ];  # instead of propagating packageA
6

This seems to be working for me, using this preliminary approach:

PackageB = 
  let pkg = poetry2nix.mkPoetryApplication {
    projectDir = self;
      propagatedBuildInputs = [ pkgs.PackageA ];
    };
  in pkg.overrideAttrs (
    prev:
      let
        partitioned = builtins.partition
          (x: (pkgs.lib.strings.hasInfix "pytest" x.name))
            prev.propagatedBuildInputs;
      in {
        propagatedBuildInputs = partitioned.wrong;
        checkInputs = prev.checkInputs ++ partitioned.right;
      }
    );

Note that this is slightly different from @wamserma’s approach in that:

  • I’m overriding propagatedBuildInputs rather than buildInputs
  • for PackageB rather than PackageA
1 Like
7

Also (I didn’t know this), you can bring in local packages (or other packages you’ve built yourself using Nix) by specifying them in the pyproject.toml, rather than by using propagatedBuildInputs as I did myself in the example above. I suspect that that’s the canonical method to do so.

8

Are there docs on how to do this?

9

Not that I’ve seen. The approach above does work, but I am still having trouble with more complex situations (ie, where PackageA shares a dependency with another package that’s deeper in the dependency graph).

10

Same here - had to change a couple of things - change pname to name, and add (final: prev: ... ) in order for poetry to accept the override.

I tried to package Zulip:
Went down a rabbit hole and kept overriding until it failed on a propagated build dep that was requiring a specific version. Ended up just using a wheel.

Thank you @wamserma for clarifying filtering!