Poetry2Nix Infinite Recursion When Depending on Pip

Help
1

While porting a project from Pip and Docker to Poetry and Nix, I am stuck when adding reportlab@3.4.0 to the dependencies.

Adding it results in the following error: 
error:
       … while evaluating the attribute 'passAsFile' of the derivation 'python3-3.8.16-env'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       … while evaluating the attribute 'passAsFile'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/build-support/buildenv/default.nix:77:5:

           76|     # XXX: The size is somewhat arbitrary
           77|     passAsFile = if builtins.stringLength pkgs >= 128*1024 then [ "pkgs" ] else [ ];
             |     ^
           78|   }

       … while evaluating call site

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/interpreters/python/wrapper.nix:20:13:

           19|   env = let
           20|     paths = requiredPythonModules (extraLibs ++ [ python ] ) ;
             |             ^
           21|     pythonPath = "${placeholder "out"}/${python.sitePackages}";

       … while calling 'requiredPythonModules'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/interpreters/python/python-packages-base.nix:48:27:

           47|   # Get list of required Python modules given a list of derivations.
           48|   requiredPythonModules = drvs: let
             |                           ^
           49|     modules = lib.filter hasPythonModule drvs;

       … while evaluating call site

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/interpreters/python/with-packages.nix:3:19:

            2|
            3| f: let packages = f pythonPackages; in buildEnv.override { extraLibs = packages; }
             |                   ^
            4|

       … while calling anonymous lambda

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/tools/poetry2nix/poetry2nix/default.nix:350:39:

          349|     in
          350|     poetryPython.python.withPackages (ps: envPkgs ++ (extraPackages ps));
             |                                       ^
          351|

       … while evaluating the attribute 'poetryPackages'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/tools/poetry2nix/poetry2nix/default.nix:282:7:

          281|       python = py;
          282|       poetryPackages = storePackages
             |       ^
          283|         ++ lib.optional hasScripts scriptsPackage

       … while evaluating call site

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/tools/poetry2nix/poetry2nix/default.nix:278:23:

          277|       */
          278|       storePackages = requiredPythonModules (builtins.foldl' (acc: v: acc ++ v) [ ] (lib.attrValues inputAttrs));
             |                       ^
          279|     in

       … while calling 'requiredPythonModules'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/interpreters/python/python-packages-base.nix:48:27:

           47|   # Get list of required Python modules given a list of derivations.
           48|   requiredPythonModules = drvs: let
             |                           ^
           49|     modules = lib.filter hasPythonModule drvs;

       … while evaluating call site

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/interpreters/python/python-packages-base.nix:65:33:

           64|         pythonPath = [ ]; # Deprecated, for compatibility.
           65|         requiredPythonModules = requiredPythonModules drv.propagatedBuildInputs;
             |                                 ^
           66|       };

       … while calling 'requiredPythonModules'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/interpreters/python/python-packages-base.nix:48:27:

           47|   # Get list of required Python modules given a list of derivations.
           48|   requiredPythonModules = drvs: let
             |                           ^
           49|     modules = lib.filter hasPythonModule drvs;

       … while evaluating call site

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/interpreters/python/python-packages-base.nix:65:33:

           64|         pythonPath = [ ]; # Deprecated, for compatibility.
           65|         requiredPythonModules = requiredPythonModules drv.propagatedBuildInputs;
             |                                 ^
           66|       };

       … while calling 'requiredPythonModules'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/interpreters/python/python-packages-base.nix:48:27:

           47|   # Get list of required Python modules given a list of derivations.
           48|   requiredPythonModules = drvs: let
             |                           ^
           49|     modules = lib.filter hasPythonModule drvs;

       … while evaluating call site

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/development/interpreters/python/python-packages-base.nix:50:6:

           49|     modules = lib.filter hasPythonModule drvs;
           50|   in lib.unique ([python] ++ modules ++ lib.concatLists (lib.catAttrs "requiredPythonModules" modules));
             |      ^
           51|

       … while calling anonymous lambda

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/lib/lists.nix:658:25:

          657|    */
          658|   unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [];
             |                         ^
          659|

       … while evaluating the attribute 'out.outPath'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/lib/customisation.nix:215:13:

          214|             drvPath = assert condition; drv.${outputName}.drvPath;
          215|             outPath = assert condition; drv.${outputName}.outPath;
             |             ^
          216|           };

       … while evaluating the attribute 'nativeBuildInputs' of the derivation 'python3.8-six-1.16.0'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       … while evaluating the attribute 'buildCommand' of the derivation 'pip-install-hook'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       … while evaluating call site

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/build-support/trivial-builders.nix:645:12:

          644|         cp ${script} $out/nix-support/setup-hook
          645|       '' + lib.optionalString (deps != []) ''
             |            ^
          646|         printWords ${toString deps} > $out/nix-support/propagated-build-inputs

       … while calling 'optionalString'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/lib/strings.nix:225:5:

          224|     # String to return if condition is true
          225|     string: if cond then string else "";
             |     ^
          226|

       … while evaluating the attribute 'outPath'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/lib/customisation.nix:222:7:

          221|       drvPath = assert condition; drv.drvPath;
          222|       outPath = assert condition; drv.outPath;
             |       ^
          223|     };

       … while evaluating the attribute 'nativeBuildInputs' of the derivation 'python3.8-pip-23.0.1'

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       error: infinite recursion encountered

       at /nix/store/h7asy5s6ibvvf8vwjkgmh3id91fppfhp-nixpkgs/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:326:7:

          325|       depsBuildBuild              = lib.elemAt (lib.elemAt dependencies 0) 0;
          326|       nativeBuildInputs           = lib.elemAt (lib.elemAt dependencies 0) 1;
             |       ^
          327|       depsBuildTarget             = lib.elemAt (lib.elemAt dependencies 0) 2;

I assume this is because installing reportlab also added a dependency on pip@23.0.1, but since I am fairly new to nix I have no idea how to debug this properly.

A few more data points about this setup:

  • I am using python38 as the python version
  • I am currently building via nixkpgs.poetry2nix.mkPoetryEnv while I’m testing things out
  • I have a few overrides set, mainly for adding setuptools or poetry as dependencies to different packages which fail to build without it.
  • I am currently using my locally installed <nixpkgs> which is 23.05pre452927.6ccc4a59c3f
1 Like
2

I’ve got this to build by adding the override

pip = nixpkgs.python38Packages.pip;

but this feels very hacky, and I hope there’s a better way.

It also results in a different pip version than the package actually required being installed (namely 22.3.1)

It also seems to absolutely ruin nix’ caching, and now all dependencies are rebuilt anytime I add one. :grimacing:

edit: The caching issue seems to have been my imagination. Directly after making this change some additions I assumed would only take little building took a lot longer than expected. Others later on seemed fine.

1 Like