What is the nix-way to specify LD_LIBRARY_PATH?

For example, I have a python package that requires libstdc++.so.6.
So my immediate solution was to

find -L /run /nix -name libstdc++.so.6
The result was (sans error messages about permission or symlink loops)
/nix/store/6ll1qmkb3ifgb0srch7j7mylab06v1d5-gcc-6.5.0-lib/lib/libstdc++.so.6
/nix/store/7fvwr8la2k701hrx2w5xnvjr5kkc7ysv-gcc-8.3.0-lib/lib/libstdc++.so.6
/nix/store/6irhcjmdd6ifnlm270y2bhp188la6n23-gcc-6.5.0-lib/lib64/libstdc++.so.6
/nix/store/6irhcjmdd6ifnlm270y2bhp188la6n23-gcc-6.5.0-lib/lib/libstdc++.so.6
/nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib64/libstdc++.so.6
/nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib/libstdc++.so.6
/nix/store/dhn13r8xx087xpyd6y3mrvl4c96ilmwb-gfortran-7.4.0-lib/lib64/libstdc++.so.6
/nix/store/dhn13r8xx087xpyd6y3mrvl4c96ilmwb-gfortran-7.4.0-lib/lib/libstdc++.so.6
Then I just ran
env LD_LIBRARY_PATH=/nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib:$LD_LIBRARY_PATH python
And problem solved.

Of course, this is not the proper nix-way. So what are the proper nix-ways if I were to
make /nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib/libstdc++.so.6 available either

  1. in a shell temporarily,
  2. system-wide by default (preferring gcc 8.3.0 over older gcc), or
  3. for a specific user by default?

Moreover, regarding how nixos works:

  1. Why libstdc++.so.6 is only available in /nix/store/*/lib but not /run/current-system/sw/lib ?
    In another occasion I used

env LD_LIBRARY_PATH=/run/current-system/sw/lib:$LD_LIBRARY_PATH

to run something requiring libX11.so.
Why libX11.so is linked under /run/current-system/sw/lib while libstdc++.so.6 is not?

  1. When I ran

find -L /nix/store/gcc-8.3.0 -name libstdc++.so.6 2>/dev/null | xargs md5sum

I got

0de7cba5b9ff44a00ddf115a1429ca36 /nix/store/7fvwr8la2k701hrx2w5xnvjr5kkc7ysv-gcc-8.3.0-lib/lib/libstdc++.so.6
a78854179583018bd6a233b3be55f78a /nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib64/libstdc++.so.6
a78854179583018bd6a233b3be55f78a /nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib/libstdc++.so.6

Why are there two different files for the gcc-8.3.0?
And why do I have two gcc-8.3.0-lib directories in /nix/store ?
(I have already executed

sudo nix-collect-garbage -d
sudo nix-store --optimise

)

4 Likes

From a very first intuitive thaught, I’d say, the proper way is to provide whatever is necessary for building as a buildInputs. propagatedBuildInputs if nix can’t recognize on its own, that the dependency indeed is used…

2 Likes

@greygreatgrape

try doing something like this

export LD_LIBRARY_PATH=$(nix eval --raw nixpkgs.zlib)/lib:$LD_LIBRARY_PATH

though Im also interested in knowing which nixpkg libstdc++.so.6 belongs to that I could include in the above variable…

1 Like

stdenv.cc.cc.lib

should do the trick.

1 Like

@danieldk

uhhhm how to do that in shell?

although I found a workaround at this thread VS Code Liveshare

 find $(nix eval --raw nixpkgs.stdenv.cc.cc.lib) -name 'libstdc++*.so*'
/nix/store/244yvw8fnxdv6zfkq2c6jqmw35jg6ib7-gcc-9.3.0-lib/lib/libstdc++.so
/nix/store/244yvw8fnxdv6zfkq2c6jqmw35jg6ib7-gcc-9.3.0-lib/lib/libstdc++.so.6.0.28-gdb.py
/nix/store/244yvw8fnxdv6zfkq2c6jqmw35jg6ib7-gcc-9.3.0-lib/lib/libstdc++.so.6
/nix/store/244yvw8fnxdv6zfkq2c6jqmw35jg6ib7-gcc-9.3.0-lib/lib/libstdc++.so.6.0.28

3 Likes

oh nice, thanks :+1:

What’s the best way to do this with Nix 2.8?

Doesn’t work:

$ nix eval --raw nixpkgs.zlib
error: experimental Nix feature 'nix-command' is disabled; use '--extra-experimental-features nix-command' to override

Also doesn’t work:

$ nix --extra-experimental-features nix-command eval --raw nixpkgs.zlib
path '/home/rkjnsn/nixpkgs.zlib' does not contain a 'flake.nix', searching up
error: getting status of '/home/rkjnsn/nixpkgs.zlib': No such file or directory

Works, but is very unwieldy:

$ nix --extra-experimental-features nix-command eval --impure --raw --expr 'with import <nixpkgs> {}; zlib'
/nix/store/fw71i85bkldrlrsr0f44741ba75j4smw-zlib-1.2.12
$ nix eval nixpkgs#zlib.outPath --raw
/nix/store/ihfwjq0djyh1x4rhh7ddmhdj9h2xhgni-zlib-1.2.12

In your case (as you obviously don’t have enabled flakes) an additional --extra-experimental-features "nix-command flakes" would be necessary.


And if you do not want to enable flakes or any other experimental feature, not even temporary, then nix-instantiate might help:

$ nix-instantiate --expr '(import <nixpkgs> {}).zlib'
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/l25hhx6c3jdgkh2vi58k3mf2fbvs0377-zlib-1.2.12.drv
3 Likes

…inside of a mkShell shellHook in a flake you can do something like

export LD_LIBRARY_PATH="${pkgs.lib.makeLibraryPath [ pkgs.pcsclite ]}:$LD_LIBRARY_PATH"
3 Likes

The nix way is not to specify LD_LIBRARY_PATH for pre-built python binaries downloaded from PyPI. You need to either:

  1. Use the pythonXXPackages. and install python packages the nix-way
  2. Do not download binary packages from PyPI and always build from source. pip install --no-binary will build and install the python package from source, using whatever is installed in your nix shell. You need to make sure all build dependencies exists and are proper nix packages. Make sure you DO NOT set LD_LIBRARY_PATH when you do pip install

To illustrate 2, here’s the greenlet python package (which gevent depends on):

[nix-shell] $ pip install greenlet
[nix-shell] $ python -c 'import greenlet'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/me/project/.venv/lib/python3.9/site-packages/greenlet/__init__.py", line 29, in <module>
    from ._greenlet import _C_API # pylint:disable=no-name-in-module
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

[nix-shell] $ pip install --force-reinstall --no-binary :all: greenlet
Collecting greenlet
  Downloading greenlet-3.0.3.tar.gz (182 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 182.0/182.0 kB 6.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: greenlet
  Building wheel for greenlet (pyproject.toml) ... done
  Created wheel for greenlet: filename=greenlet-3.0.3-cp311-cp311-linux_aarch64.whl size=595251 sha256=0a05212e1a6c202825becf9a94778557c40c3c98f6490c6a404038b484977f27
  Stored in directory: /home/me/.cache/pip/wheels/56/08/ca/e0bd72a4cd850a40d2a51ee67fbd2f3efe5332fe5fe00e2c53
Successfully built greenlet
Installing collected packages: greenlet
Successfully installed greenlet-3.0.3

[nix-shell]$ python -c 'import greenlet; print("success");'
success
5 Likes