Why some Python package comes with PYTHONPATH and another not?

imincik · September 30, 2021, 5:48pm

Hi all,

when building my nix-shell environment, I found that including package pre-commit in mkShell buildInputs results in populated PYTHONPATH shell environment variable, but including Poetry doesn’t. Both are Python packages and are built in very similar way. The only difference is that Poetry declares pythonNamespaces while pre-commit doesn’t, but according my testing (trying to override pre-commit with setting pythonNamespaces) it doesn’t have any impact on resulting PYTHONPATH.

Example:

shell.nix with poetry:

    pkgs.mkShell {
        name = "xyz";
        buildInputs = with pkgs; [
            poetry
        ];

$ echo $PYTHONPATH
[NOTHING]

shell.nix with pre-commit:

    pkgs.mkShell {
        name = "xyz";
        buildInputs = with pkgs; [
            pre-commit
        ];

$ echo $PYTHONPATH

/nix/store/6kcxf8w39r1gfgbssg8jwigsg4bz21xw-python3.8-pre-commit-2.11.0/lib/python3.8/site-packages:/nix/store/k0x66mivh0dmmiw68xx4qnkcakvdjcw9-python3.8-aspy.yaml-1.3.0/lib/python3.8/site-packages:/nix/store/hqfbrk1q1zjj04990r3frhnxrqb1by8g-python3.8-PyYAML-5.4.1/lib/python3.8/site-packages: ....

Thank you very much.

imincik · September 30, 2021, 6:20pm

Looks like it might be PYTHOPATH leaking as described in some other threads but I am still not sure what is causing this especially in this case when one package leaks and other not.

jtojnar · September 30, 2021, 6:21pm

You are looking at pkgs/development/python-modules/poetry/default.nix (used by pkgs.python3.pkgs.poetry) but pkgs.poetry points to pkgs/development/tools/poetry2nix/poetry2nix/pkgs/poetry/default.nix:

github.com

NixOS/nixpkgs/blob/c21ba4f7bb4a3d621eb1d187e6b5e816bb85380c/pkgs/top-level/all-packages.nix#L13137

    
      
          
          
# These pyside tools do not provide any Python modules and are meant to be here.
          # See ../development/python-modules/pyside/default.nix for details.
          pysideApiextractor = callPackage ../development/python-modules/pyside/apiextractor.nix { };
          pysideGeneratorrunner = callPackage ../development/python-modules/pyside/generatorrunner.nix { };
          
          
svg2tikz = python27Packages.svg2tikz;
          
          
pew = callPackage ../development/tools/pew {};
          
          
poetry = callPackage ../development/tools/poetry2nix/poetry2nix/pkgs/poetry {
            python = python3;
          };
          poetry2nix = callPackage ../development/tools/poetry2nix/poetry2nix {
            inherit pkgs lib;
          };
          
          
poetry2conda = python3Packages.callPackage ../development/python-modules/poetry2conda { };
          
          
pipenv = callPackage ../development/tools/pipenv {};

imincik · September 30, 2021, 7:18pm

If I understand it correctly, poetry is now built by poetry2nix (pkgs/development/tools/poetry2nix/poetry2nix/pkgs/poetry/default.nix) and pkgs/development/python-modules/poetry/default.nix is not used at all for (there is no reference of it in all-packages.nix) ?