The Context
One day I will be cool enough to just fix my team’s dependencies so that I can use poetry2nix for everything. Today is not that day.
Instead I’m using a flake devshell for things that poetry can’t handle (like poetry itself) and I’m using poetry for everything that poetry can handle (like fastapi). This usually works. poetry run is invoked from the devshell, so I end up with both sets of dependencies.
The Problem
I think that something from the devshell is conflicting with something in poetry. No tests run if I use the devshell, because there’s a conflict in conftest.py
❯ nix develop
$ poetry run pytest
ImportError while loading conftest '/Users/matt/src/myproj/tests/conftest.py'.
tests/conftest.py:14: in <module>
from fastapi.testclient import TestClient
...
../../Library/Caches/pypoetry/virtualenvs/myproj-R0lYn3as-py3.11/lib/python3.11/site-packages/fastapi/exceptions.py:6: in <module>
from typing_extensions import Annotated, Doc # type: ignore [attr-defined]
E ImportError: cannot import name 'Doc' from 'typing_extensions' (/nix/store/0wi4xxrgz86n5c8n8snaq7wl5mmc229n-python3.11-typing-extensions-4.7.1/lib/python3.11/site-packages/typing_extensions.py)
It works if I exit the devshell… until tests fail due to missing dependencies
❯ poetry run pytest -s -vv 'tests'
... several tests pass ...
... a test fails because `ruff` is missing from the environment (defined in the flake, which is now ignored)...
The problem appeared when fastapi updated. There’s a conversation about it here: Module 'Doc' import error when using FastAPI · tiangolo/fastapi · Discussion #10476 · GitHub, many people are having this problem in google collab. I bet that the conflict that’s present in collab is also in my devshell.
The Question
This is the path from the error:
/nix/store/0wi4xxrgz86n5c8n8snaq7wl5mmc229n-python3.11-typing-extensions-4.7.1/lib/python3.11/site-packages/typing_extensions.py
How can I trace it back to a package in my devshell which it is presumably conflicting with?
Candidates are:
devShells.default = pkgs.mkShell {
packages = [
(pkgs.poetry.overrideAttrs (previousAttrs: {
makeWrapperArgs = ''
--suffix PYTHONPATH : .
'';
}))
deps.kustomize
pkgs.python311
pkgs.python311Packages.python-lsp-server
pkgs.python311Packages.python-lsp-ruff
pkgs.python311Packages.pylsp-rope
pkgs.python311Packages.pylsp-mypy
pkgs.python311Packages.pudb
(pkgs.pre-commit.overrideAttrs (previousAttrs: {
doCheck = false;
makeWrapperArgs = ''
--prefix PYTHONPATH : .
--prefix PYTHONPATH : $PYTHONPATH
--prefix PYTHONPATH : ${pkgs.python311Packages.python}/${pkgs.python311Packages.python.sitePackages}
--prefix PYTHONPATH : ${pkgs.python311Packages.ruamel-yaml}/lib/python3.11/site-packages
'';
}))
pkgs.python311Packages.psycopg
#pkgs.python311Packages.greenlet # for locust, which has cpp deps
#pkgs.python311Packages.mitmproxy
pkgs.ruff
pkgs.just
pkgs.jq
pkgs.teleport
pkgs.kubectl
pkgs.wget
pkgs.tilt
pkgs.kind
pkgs.circleci-cli
pkgs.openssl
pkgs.yq-go
pkgs.actionlint
];
nativeBuildInputs = (if pkgs.stdenv.isLinux then [ pkgs.autoPatchelfHook ] else []);
shellHook = ''
# pip builds ruff in a way that references some dependencies implicitly by filepath
# this breaks on NixOS which likes to be explicit about such things
# https://github.com/astral-sh/ruff-pre-commit/issues/22#issuecomment-1684170057
if cat /etc/os-release | grep NixOS &> /dev/null ; then
if [ -d ~/.cache/pre-commit ] ; then
patch=$(autoPatchelf ~/.cache/pre-commit/)
if [[ $? -ne 0 ]]; then
echo "$patch"
exit 1
else
echo "patched ~/.cache/pre-commit"
fi
fi
fi
export KIND_CLUSTER_IMAGE='kindest/node:v1.25.3'
export export PYTHONBREAKPOINT="pudb.set_trace"
'';
};