What is the nix-way to specify LD_LIBRARY_PATH?

greygreatgrape · March 25, 2020, 5:00am

For example, I have a python package that requires libstdc++.so.6.
So my immediate solution was to

find -L /run /nix -name libstdc++.so.6
The result was (sans error messages about permission or symlink loops)
/nix/store/6ll1qmkb3ifgb0srch7j7mylab06v1d5-gcc-6.5.0-lib/lib/libstdc++.so.6
/nix/store/7fvwr8la2k701hrx2w5xnvjr5kkc7ysv-gcc-8.3.0-lib/lib/libstdc++.so.6
/nix/store/6irhcjmdd6ifnlm270y2bhp188la6n23-gcc-6.5.0-lib/lib64/libstdc++.so.6
/nix/store/6irhcjmdd6ifnlm270y2bhp188la6n23-gcc-6.5.0-lib/lib/libstdc++.so.6
/nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib64/libstdc++.so.6
/nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib/libstdc++.so.6
/nix/store/dhn13r8xx087xpyd6y3mrvl4c96ilmwb-gfortran-7.4.0-lib/lib64/libstdc++.so.6
/nix/store/dhn13r8xx087xpyd6y3mrvl4c96ilmwb-gfortran-7.4.0-lib/lib/libstdc++.so.6
Then I just ran
env LD_LIBRARY_PATH=/nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib:$LD_LIBRARY_PATH python
And problem solved.

Of course, this is not the proper nix-way. So what are the proper nix-ways if I were to
make /nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib/libstdc++.so.6 available either

in a shell temporarily,
system-wide by default (preferring gcc 8.3.0 over older gcc), or
for a specific user by default?

Moreover, regarding how nixos works:

Why libstdc++.so.6 is only available in /nix/store/*/lib but not /run/current-system/sw/lib ?
In another occasion I used

env LD_LIBRARY_PATH=/run/current-system/sw/lib:$LD_LIBRARY_PATH

to run something requiring libX11.so.
Why libX11.so is linked under /run/current-system/sw/lib while libstdc++.so.6 is not?

When I ran

find -L /nix/store/gcc-8.3.0 -name libstdc++.so.6 2>/dev/null | xargs md5sum

I got

0de7cba5b9ff44a00ddf115a1429ca36 /nix/store/7fvwr8la2k701hrx2w5xnvjr5kkc7ysv-gcc-8.3.0-lib/lib/libstdc++.so.6
a78854179583018bd6a233b3be55f78a /nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib64/libstdc++.so.6
a78854179583018bd6a233b3be55f78a /nix/store/784rh7jrfhagbkydjfrv68h9x3g4gqmk-gcc-8.3.0-lib/lib/libstdc++.so.6

Why are there two different files for the gcc-8.3.0?
And why do I have two gcc-8.3.0-lib directories in /nix/store ?
(I have already executed

sudo nix-collect-garbage -d
sudo nix-store --optimise

)

NobbZ · March 25, 2020, 7:44pm

From a very first intuitive thaught, I’d say, the proper way is to provide whatever is necessary for building as a buildInputs. propagatedBuildInputs if nix can’t recognize on its own, that the dependency indeed is used…

macalimlim · May 14, 2020, 3:26pm

@greygreatgrape

try doing something like this

export LD_LIBRARY_PATH=$(nix eval --raw nixpkgs.zlib)/lib:$LD_LIBRARY_PATH

though Im also interested in knowing which nixpkg libstdc++.so.6 belongs to that I could include in the above variable…

danieldk · May 14, 2020, 3:37pm

stdenv.cc.cc.lib

should do the trick.

macalimlim · May 14, 2020, 3:41pm

@danieldk

uhhhm how to do that in shell?

although I found a workaround at this thread VS Code Liveshare

danieldk · May 14, 2020, 3:42pm

 find $(nix eval --raw nixpkgs.stdenv.cc.cc.lib) -name 'libstdc++*.so*'
/nix/store/244yvw8fnxdv6zfkq2c6jqmw35jg6ib7-gcc-9.3.0-lib/lib/libstdc++.so
/nix/store/244yvw8fnxdv6zfkq2c6jqmw35jg6ib7-gcc-9.3.0-lib/lib/libstdc++.so.6.0.28-gdb.py
/nix/store/244yvw8fnxdv6zfkq2c6jqmw35jg6ib7-gcc-9.3.0-lib/lib/libstdc++.so.6
/nix/store/244yvw8fnxdv6zfkq2c6jqmw35jg6ib7-gcc-9.3.0-lib/lib/libstdc++.so.6.0.28

macalimlim · May 14, 2020, 3:44pm

oh nice, thanks

rkjnsn · June 16, 2022, 9:20am

What’s the best way to do this with Nix 2.8?

Doesn’t work:

$ nix eval --raw nixpkgs.zlib
error: experimental Nix feature 'nix-command' is disabled; use '--extra-experimental-features nix-command' to override

Also doesn’t work:

$ nix --extra-experimental-features nix-command eval --raw nixpkgs.zlib
path '/home/rkjnsn/nixpkgs.zlib' does not contain a 'flake.nix', searching up
error: getting status of '/home/rkjnsn/nixpkgs.zlib': No such file or directory

Works, but is very unwieldy:

$ nix --extra-experimental-features nix-command eval --impure --raw --expr 'with import <nixpkgs> {}; zlib'
/nix/store/fw71i85bkldrlrsr0f44741ba75j4smw-zlib-1.2.12

NobbZ · June 16, 2022, 10:59am

$ nix eval nixpkgs#zlib.outPath --raw
/nix/store/ihfwjq0djyh1x4rhh7ddmhdj9h2xhgni-zlib-1.2.12

In your case (as you obviously don’t have enabled flakes) an additional --extra-experimental-features "nix-command flakes" would be necessary.

And if you do not want to enable flakes or any other experimental feature, not even temporary, then nix-instantiate might help:

$ nix-instantiate --expr '(import <nixpkgs> {}).zlib'
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/l25hhx6c3jdgkh2vi58k3mf2fbvs0377-zlib-1.2.12.drv

samrose · May 8, 2024, 3:02pm

…inside of a mkShell shellHook in a flake you can do something like

export LD_LIBRARY_PATH="${pkgs.lib.makeLibraryPath [ pkgs.pcsclite ]}:$LD_LIBRARY_PATH"

9gel · July 1, 2024, 9:26am

The nix way is not to specify LD_LIBRARY_PATH for pre-built python binaries downloaded from PyPI. You need to either:

Use the pythonXXPackages. and install python packages the nix-way
Do not download binary packages from PyPI and always build from source. pip install --no-binary will build and install the python package from source, using whatever is installed in your nix shell. You need to make sure all build dependencies exists and are proper nix packages. Make sure you DO NOT set LD_LIBRARY_PATH when you do pip install

To illustrate 2, here’s the greenlet python package (which gevent depends on):

[nix-shell] $ pip install greenlet
[nix-shell] $ python -c 'import greenlet'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/me/project/.venv/lib/python3.9/site-packages/greenlet/__init__.py", line 29, in <module>
    from ._greenlet import _C_API # pylint:disable=no-name-in-module
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

[nix-shell] $ pip install --force-reinstall --no-binary :all: greenlet
Collecting greenlet
  Downloading greenlet-3.0.3.tar.gz (182 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 182.0/182.0 kB 6.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: greenlet
  Building wheel for greenlet (pyproject.toml) ... done
  Created wheel for greenlet: filename=greenlet-3.0.3-cp311-cp311-linux_aarch64.whl size=595251 sha256=0a05212e1a6c202825becf9a94778557c40c3c98f6490c6a404038b484977f27
  Stored in directory: /home/me/.cache/pip/wheels/56/08/ca/e0bd72a4cd850a40d2a51ee67fbd2f3efe5332fe5fe00e2c53
Successfully built greenlet
Installing collected packages: greenlet
Successfully installed greenlet-3.0.3

[nix-shell]$ python -c 'import greenlet; print("success");'
success