Nix-ld does not work with virtualenv

Help
1

I was expecting enabling nix-ld to be enough to run basically anything like in any other system, as it was working fine with npm etc. But I tried today with virtualenv (for some reasons I can’t manage to run some python packages), and I was surprised to see that it is failing:

$ echo $NIX_LD
/nix/store/w7982l1alnpynwh3mlf6ibz44b1y9ypj-ld.so
$ echo $NIX_LD_LIBRARY_PATH 
/run/current-system/sw/share/nix-ld/lib
$ nix-shell -p (python3.withPackages (ps: with ps; [ virtualenv ]))
$ virtualenv venv
$ source ./venv/bin/activate
$ pip install matplotlib
$ python
Python 3.10.9 (main, Dec  6 2022, 18:44:57) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/python/venv/lib/python3.10/site-packages/matplotlib/__init__.py", line 161, in <module>
    from . import _api, _version, cbook, _docstring, rcsetup
  File "/tmp/python/venv/lib/python3.10/site-packages/matplotlib/rcsetup.py", line 27, in <module>
    from matplotlib.colors import Colormap, is_color_like
  File "/tmp/python/venv/lib/python3.10/site-packages/matplotlib/colors.py", line 57, in <module>
    from matplotlib import _api, _cm, cbook, scale
  File "/tmp/python/venv/lib/python3.10/site-packages/matplotlib/scale.py", line 22, in <module>
    from matplotlib.ticker import (
  File "/tmp/python/venv/lib/python3.10/site-packages/matplotlib/ticker.py", line 143, in <module>
    from matplotlib import transforms as mtransforms
  File "/tmp/python/venv/lib/python3.10/site-packages/matplotlib/transforms.py", line 49, in <module>
    from matplotlib._path import (
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory
>>>

I tried to check, libstdc is indeed present:

$ ls $NIX_LD_LIBRARY_PATH | rg libstdc
libstdc++fs.la
libstdc++.la
libstdc++.so
libstdc++.so.6
libstdc++.so.6.0.29
libstdc++.so.6.0.29-gdb.py

So now my guess is that python relies on a different loader than the one controlled by nix-ld, causing the above issue as this one is not allowed to run. Doing:

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$NIX_LD_LIBRARY_PATH"

seems to solve the problem, but I know that LD_LIBRARY_PATH is not really recommended here (or maybe there is a better alternative variable to use?)… is there any other, cleaner solution to support any transparent pip development?

My full nix-ld config:

  programs.nix-ld = {
    enable = true;
    libraries = with pkgs; [
      # List by default
      zlib
      zstd
      stdenv.cc.cc
      curl
      openssl
      attr
      libssh
      bzip2
      libxml2
      acl
      libsodium
      util-linux
      xz
      systemd

      # My own additions
	    xorg.libXcomposite
      xorg.libXtst
      xorg.libXrandr
      xorg.libXext
      xorg.libX11
      xorg.libXfixes
      libGL
      libva
      pipewire.lib
      xorg.libxcb
      xorg.libXdamage
      xorg.libxshmfence
      xorg.libXxf86vm
      libelf
      
      # Required
      glib
      gtk2
      
      # Without these it silently fails
      xorg.libXinerama
      xorg.libXcursor
      xorg.libXrender
      xorg.libXScrnSaver
      xorg.libXi
      xorg.libSM
      xorg.libICE
      gnome2.GConf
      nspr
      nss
      cups
      libcap
      SDL2
      libusb1
      dbus-glib
      ffmpeg
      # Only libraries are needed from those two
      libudev0-shim

      # needed to run unity
      gtk3
      icu
      libnotify
      gsettings-desktop-schemas
      # https://github.com/NixOS/nixpkgs/issues/72282
      # https://github.com/NixOS/nixpkgs/blob/2e87260fafdd3d18aa1719246fd704b35e55b0f2/pkgs/applications/misc/joplin-desktop/default.nix#L16
      # log in /home/leo/.config/unity3d/Editor.log
      # it will segfault when opening files if you don’t do:
      # export XDG_DATA_DIRS=/nix/store/0nfsywbk0qml4faa7sk3sdfmbd85b7ra-gsettings-desktop-schemas-43.0/share/gsettings-schemas/gsettings-desktop-schemas-43.0:/nix/store/rkscn1raa3x850zq7jp9q3j5ghcf6zi2-gtk+3-3.24.35/share/gsettings-schemas/gtk+3-3.24.35/:$XDG_DATA_DIRS
      # other issue: (Unity:377230): GLib-GIO-CRITICAL **: 21:09:04.706: g_dbus_proxy_call_sync_internal: assertion 'G_IS_DBUS_PROXY (proxy)' failed
      
      # Verified games requirements
      xorg.libXt
      xorg.libXmu
      libogg
      libvorbis
      SDL
      SDL2_image
      glew110
      libidn
      tbb
      
      # Other things from runtime
      flac
      freeglut
      libjpeg
      libpng
      libpng12
      libsamplerate
      libmikmod
      libtheora
      libtiff
      pixman
      speex
      SDL_image
      SDL_ttf
      SDL_mixer
      SDL2_ttf
      SDL2_mixer
      libappindicator-gtk2
      libdbusmenu-gtk2
      libindicator-gtk2
      libcaca
      libcanberra
      libgcrypt
      libvpx
      librsvg
      xorg.libXft
      libvdpau
      # ...
      # Some more libraries that I needed to run programs
      gnome2.pango
      cairo
      atk
      gdk-pixbuf
      fontconfig
      freetype
      dbus
      alsaLib
      expat
      # Needed for electron
      libdrm
      mesa
      libxkbcommon
      # Needed to run, via virtualenv + pip, matplotlib & tikzplotlib
      stdenv.cc.cc.lib # to provide libstdc++.so.6
    ];
  };
2

I think this FAQ entry should answer your question.

Basically you are using a python binary from nixpkg that uses the correct linker and therefore is not affected by nix-ld.

3

Thanks! But what I don’t like with NIX_LD_LIBRARY_PATH is that it will take priority over the entries in RPATH, that might come issues. Do you know if other variables/path can be used? I tried to list all libraries in /etc/ld.so.cache and run sudo ldconfig -C /etc/ld.so.cache -f /etc/ld.so.conf without success (I used sudo ldconfig -C /etc/ld.so.cache -f /etc/ld.so.conf, no error, but I think that the loader is not reading this ld.so.cache folder).

1 Like
4

You could try GitHub - GuillaumeDesforges/fix-python: A simple CLI tool to allow you to use Python "normally" in NixOS, which was made for exactly that use case :slight_smile:

2 Likes