The nixpkgs manual shows how it’s possible to create an impure python env, in which some python packages, e.g. matplotlib, are installed via Nix (for example, because they include C libraries that depend on glibc), whereas others are installed impurely with pip.
I’m trying to do the same, but with poetry, but this doesn’t seem to work: the dependencies installed via nix are not visible to a python interpreter run under poetry (poetry run python), as poetry run removes from sys.path the derivation containing the deps installed via nix.
Below is a repro case.
I have the following flake.nix:
{
description = "A very basic flake";
outputs = { self, nixpkgs }: {
devShells.x86_64-linux.default = let pkgs = nixpkgs.legacyPackages.x86_64-linux; in
pkgs.mkShell {
name = "test";
packages = with pkgs; [
poetry
(python3.withPackages (ps: with ps; [ matplotlib ]))
];
};
};
}
and a stock pyproject.toml:
[tool.poetry]
name = "foo"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.10"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
In a nix develop shell, the following works:
$ python -c "import matplotlib"
$
whereas this fails:
$ poetry run python -c "import matplotlib"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'matplotlib'
$
Looking at the output of python -m site, there’s a discrepancy:
$ python -m site
sys.path = [
'/home/asymmetric/code/foo',
'/nix/store/6qk2ybm2yx2dxmx9h4dikr1shjhhbpfr-python3-3.10.11/lib/python310.zip',
'/nix/store/6qk2ybm2yx2dxmx9h4dikr1shjhhbpfr-python3-3.10.11/lib/python3.10',
'/nix/store/6qk2ybm2yx2dxmx9h4dikr1shjhhbpfr-python3-3.10.11/lib/python3.10/lib-dynload',
'/nix/store/6qk2ybm2yx2dxmx9h4dikr1shjhhbpfr-python3-3.10.11/lib/python3.10/site-packages',
'/nix/store/qyy1khnfvvv6m57gkh8qijj45v3h78jy-python3-3.10.11-env/lib/python3.10/site-packages',
]
USER_BASE: '/home/asymmetric/.local' (exists)
USER_SITE: '/home/asymmetric/.local/lib/python3.10/site-packages' (doesn't exist)
ENABLE_USER_SITE: False
versus
$ poetry run python -m site
sys.path = [
'/home/asymmetric/code/foo',
'/nix/store/6qk2ybm2yx2dxmx9h4dikr1shjhhbpfr-python3-3.10.11/lib/python310.zip',
'/nix/store/6qk2ybm2yx2dxmx9h4dikr1shjhhbpfr-python3-3.10.11/lib/python3.10',
'/nix/store/6qk2ybm2yx2dxmx9h4dikr1shjhhbpfr-python3-3.10.11/lib/python3.10/lib-dynload',
'/home/asymmetric/code/foo/.venv/lib/python3.10/site-packages',
'/nix/store/6qk2ybm2yx2dxmx9h4dikr1shjhhbpfr-python3-3.10.11/lib/python3.10/site-packages',
]
USER_BASE: '/home/asymmetric/.local' (exists)
USER_SITE: '/home/asymmetric/.local/lib/python3.10/site-packages' (doesn't exist)
ENABLE_USER_SITE: False
Note that in the second case, the -env derivation is missing, which is the one with the Python environment containing matplotlib. (Also, poetry automatically creates a venv, but I think that’s not relevant for this problem)
Is there a way to add the missing paths to sys.path?