I’ve been using nix for ~2 years full time: on my machines including a personal laptop, and in projects.
All the projects I work on have flake.nix, and I’d say I’m somewhere around 7/10 with nix familiarity.
Sometimes, I still run into stuff with no idea of how to approach the problem, like this one:
Python has deprecated it’s utcnow and this made some tests failing in deps that are using utcnow, like babel.
The projects I’m working on ATM are using poetry2nix to manage Python dev enironment.
After updating python to 3.12, I can’t enter a devshell anymore, because there’s a babel-2.12.1 failing checkPhase:
error: builder for '/nix/store/xaf8yxfph37jsyjira2rw6namya43saz-python3.12-babel-2.12.1.drv' failed with exit code 1;
last 10 log lines:
> tests/test_dates.py::test_format_current_moment
> /build/Babel-2.12.1/tests/test_dates.py:705: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
> frozen_instant = datetime.utcnow()
>
> -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
> =========================== short test summary info ============================
> FAILED tests/messages/test_extract.py::ExtractTestCase::test_f_strings - AssertionError: assert 3 == 4
> FAILED tests/messages/test_extract.py::ExtractTestCase::test_f_strings_non_utf8 - assert 0 == 1
> ==== 2 failed, 4887 passed, 6 skipped, 5 deselected, 301 warnings in 7.82s =====
Now, the problem is figuring out how babel is getting propagated into dependency graph.
I’ve tried:
excluding all deps and adding 1 by 1, but as soon as I add any dependency to pyproject.toml and poetry lock, babel gets in and fails
explicitly requiring updated babel 2.13 in pyproject.toml, but no matter of what, python 3.12 still builds babel-2.12.1
scanning poetry.lock for babel
looking into nix build logs to determine what requires it
adding an overlay in flake.nix with doCheck = false on babel, no luck but also want to not have babel-2.12.1 in the dep graph if it’s broken
Not looking for hand holding or help here, just putting this out there because it sometimes baffles me how hard is it to resolve an issue like this one. Is there a good link with advanced debugging techniques with nix that would help in this case?
I’m not looking for answers like “bump poetry in your flake.nix”, but rather “this is how to approach these kinds of problems with nix”.
I use nix path-info --recursive on a derivation to see what it depends on, then nix why-depends in order to trace … well, why it depends on that thing.
$ nix why-depends .#devShells.x86_64-linux.default nixpkgs#python312Packages.babel squale-capital/clenow
warning: Git tree '/home/supermarin/code/squale-capital/clenow' is dirty
error: build of '/nix/store/xaf8yxfph37jsyjira2rw6namya43saz-python3.12-babel-2.12.1.drv' on 'ssh://mx-001' failed: builder for '/nix/store/xaf8yxfph37jsyjira2rw6namya43saz-python3.12-babel-2.12.1.drv' failed with exit code 1
error: builder for '/nix/store/xaf8yxfph37jsyjira2rw6namya43saz-python3.12-babel-2.12.1.drv' failed with exit code 1;
last 10 log lines:
> tests/test_dates.py::test_format_current_moment
> /build/Babel-2.12.1/tests/test_dates.py:705: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
> frozen_instant = datetime.utcnow()
>
> -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
> =========================== short test summary info ============================
> FAILED tests/messages/test_extract.py::ExtractTestCase::test_f_strings - AssertionError: assert 3 == 4
> FAILED tests/messages/test_extract.py::ExtractTestCase::test_f_strings_non_utf8 - assert 0 == 1
> ==== 2 failed, 4887 passed, 6 skipped, 5 deselected, 301 warnings in 7.86s =====
> /nix/store/wr08yanv2bjrphhi5aai12hf2qz5kvic-stdenv-linux/setup: line 1559: pop_var_context: head of shell_variables not a function context
For full logs, run 'nix log /nix/store/xaf8yxfph37jsyjira2rw6namya43saz-python3.12-babel-2.12.1.drv'.
error: 1 dependencies of derivation '/nix/store/2am3zlkrx88l9m31q8mk3xih35sx6lnx-python3.12-sphinx-7.2.6.drv' failed to build
error: 1 dependencies of derivation '/nix/store/97h4z1cwpfsdkdvrzjinyy0xri2apf1f-python3.12-pip-23.2.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/001bqiq7yl2fwlsi5542fv2jczsagjj6-pip-build-hook.sh.drv' failed to build
error: 1 dependencies of derivation '/nix/store/aa4gni4v9m1ad1hhcfkn7qwwpwvbz7zm-python3.12-astroid-3.0.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/vb2rgfsq7g7ng8g404bc8937ch7awa8f-python3.12-pathspec-0.11.2.drv' failed to build
error: 1 dependencies of derivation '/nix/store/02cv70nhmqglmpb77gwd90xs6yi6sx48-python3.12-platformdirs-4.1.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/38wk25am4aa5mdyl7lwkwgp24m1ssg81-python3.12-pluggy-1.3.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/vrx7qjwdv1ll7dmc817hyzwphdlc2pyr-python3.12-pytest-7.4.3.drv' failed to build
error: 1 dependencies of derivation '/nix/store/5d26mljdf4d7hzlwnaix9pskliz7wnyn-python3-3.12.0-env.drv' failed to build
error: 1 dependencies of derivation '/nix/store/g6328bgy16xdl5ky03zwjg0mnzd1lx76-interactive-python3-3.12.0-environment.drv' failed to build
I tried to grep the output of nix path-info --recursive, which prints a list of derivations, and found babel mentioned in jinja2 and sphinx. Both are propagated deps in my case.
Tried removing babel from propagatedBuildInputs like below, but it still gets built.
@vcunat thank you, great pointer. It’s been a while since last time I used why-depends, missed the --derivation flag. IMO the tool should print both runtime and build-time deps and only filter out with a flag instead of only runtime by default - the UX is pretty confusing.
Now back to the issue, as expected this is a propagated dep of a dep of a dep… of python itself.