Problems with poetry2nix and matplotlib

Help
1

I am just starting out in developing a workflow in python to do mostly systems engineering and data science.
I found poetry2nix and chose this solution since it seemed to be the best of all worlds. Portable environment with poetry which manages all the python packages and then a flake to manage other dependencies and integrate well with nix.
However when importing matplotlib i get the following error:

>>> import matplotlib.pyplot as plt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/marci/.cache/pypoetry/virtualenvs/app-ZnXbtN1Y-py3.11/lib/python3.11/site-packages/matplotlib/__init__.py", line 161, in <module>
    from . import _api, _version, cbook, _docstring, rcsetup
  File "/home/marci/.cache/pypoetry/virtualenvs/app-ZnXbtN1Y-py3.11/lib/python3.11/site-packages/matplotlib/rcsetup.py", line 27, in <module>
    from matplotlib.colors import Colormap, is_color_like
  File "/home/marci/.cache/pypoetry/virtualenvs/app-ZnXbtN1Y-py3.11/lib/python3.11/site-packages/matplotlib/colors.py", line 57, in <module>
    from matplotlib import _api, _cm, cbook, scale
  File "/home/marci/.cache/pypoetry/virtualenvs/app-ZnXbtN1Y-py3.11/lib/python3.11/site-packages/matplotlib/scale.py", line 22, in <module>
    from matplotlib.ticker import (
  File "/home/marci/.cache/pypoetry/virtualenvs/app-ZnXbtN1Y-py3.11/lib/python3.11/site-packages/matplotlib/ticker.py", line 143, in <module>
    from matplotlib import transforms as mtransforms
  File "/home/marci/.cache/pypoetry/virtualenvs/app-ZnXbtN1Y-py3.11/lib/python3.11/site-packages/matplotlib/transforms.py", line 49, in <module>
    from matplotlib._path import (
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

Steps to reproduce:

  1. nix flake init --template github:nix-community/poetry2nix
  2. nix develop
  3. poetry add matplotlib
  4. poetry run python
  5. import matplotlib.plot as plt

I tried to fix it mostly with this thread but could not get it to work with any of the proposed solutions. The same error still persists.
If anyone has any experience in using python with nix or poetry2nix I would be very grateful for tips on how to fix this problem and what are general best practices when working with these tools.

I am on recent NixOS 23.11.

2

poetry2nix reads the poetry.lock file which is generated by the poetry lock command, so if you want it to do anything you have to (re)generate that file before (re)entering your devshell.

Try this:

  1. nix flake init --template github:nix-community/poetry2nix
  2. nix develop
  3. poetry add matplotlib
  4. poetry lock # pins versions, writes them to poetry.lock
  5. exit # the devshell gets built on entry, so lets leave and come back
  6. nix develop # may take a while while it builds stuff for numpy, etc
  7. poetry run python
  8. import matplotlib.pyplot as plt

Also, a word of caution: poetry2nix is pretty cool, but it’s not magic. There are packages out there which will break when you use them through poetry2nix. Usually this is because the package is doing something abnormal which poetry2nix hasn’t anticipated: Python culture is permissive like that, for better or worse.

You can add special-case code for that package to poetry2nix, thereby fixing it for everybody, but my nix was not strong enough and I got stuck at this phase. I plan to take a crack at it again soon.

Matplotlib is the sort of thing that is likely to require extra care with poetry2nix, since it has c++ dependencies, but it’s popular enough that somebody has already added the special case code for it. So if you stick to popular libraries, you may be able to avoid adding them yourself.

3

Hi thanks for the reply but unfortunately this results in the same problem.

At step 6. it does not build anything. It does state on the poetry2nix github that building from source should be the default, but it seems to be downloading binary packages when running poetry add matplotlib. Is there another workflow/task I am missing?

4

For now I switched to declaring the python packages directly in the poetry2nix flake and running through this, which works until I need some more obscure python modules probably.
However this is not optimal for collaboration, and I would also like to figure out the right approach with poetry2nix.