How to solve libstdc++ not found in shell.nix

Help
1

Hello,

I am trying to run nix-shell to use poetry (and hopefully poetry2nix). I am stuck with an error from an executable from a python package that claims libstdc++ is missing.

I’ve found something that looks like a solution on Packaging/Quirks and Caveats - NixOS Wiki (linked from What package provides libstdc++.so.6?)
but it is for packaging and not nix-shell.

With nix-shell I get the error

error: undefined variable 'stdenv'

       at /home/espen/code/python-fmri-env/shell.nix:11:23:

           10|     # fixes libstdc++ issues
           11|     LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/
             |                       ^
           12|     '';

So how to I refer to this variable in shell.nix? I’ve tried to add stdenv like { stdenv, pkgs ? import … } but no luck. I need it to work both on Nix and on NixOS.

shell.nix:

{ pkgs ? import <nixpkgs> {} }:

pkgs.mkShell {

  buildInputs = [
    pkgs.python310
    pkgs.poetry
  ];
  shellHook = ''
    # fixes libstdc++ issues and libgl.so issues
    LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/
  ''
}

EDIT: clarity and mistakes

1 Like
2

stdenv is in pkgs, so just use pkgs.stdenv.cc.cc.lib.

3 Likes
3

Thank you!

shell.nix now evaluates, but unfortunately I still get the same error. Seems like the solution I linked to wasn’t enough.

The error message I get is:

(python-fmri-env-py3.10) espen@mainframe-ubuntu:~/code/python-fmri-env$ clinica
Traceback (most recent call last):
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/bin/clinica", line 5, in <module>
    from clinica.cmdline import main
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/clinica/cmdline.py", line 13, in <module>
    from clinica.iotools.converters.cli import cli as convert_cli
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/clinica/iotools/converters/cli.py", line 5, in <module>
    from .habs_to_bids import habs_to_bids_cli
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/clinica/iotools/converters/habs_to_bids/habs_to_bids_cli.py", line 5, in <module>
    import clinica.iotools.bids_utils as bids
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/clinica/iotools/bids_utils.py", line 7, in <module>
    from pandas import DataFrame
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/pandas/__init__.py", line 48, in <module>
    from pandas.core.api import (
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/pandas/core/api.py", line 47, in <module>
    from pandas.core.groupby import (
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import (
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 76, in <module>
    from pandas.core.frame import DataFrame
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/pandas/core/frame.py", line 172, in <module>
    from pandas.core.generic import NDFrame
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/pandas/core/generic.py", line 169, in <module>
    from pandas.core.window import (
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/pandas/core/window/__init__.py", line 1, in <module>
    from pandas.core.window.ewm import (
  File "/home/espen/.cache/pypoetry/virtualenvs/python-fmri-env-5taD3dGB-py3.10/lib/python3.10/site-packages/pandas/core/window/ewm.py", line 15, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

I get the same error both with Ubuntu/Nix and NixOS. Any ideas how to fix this?

4

it might be because that LD_LIBRARY_PATH is not exported

2 Likes
5

Amazing, that works! Good catch!

6

Just a word of warning.

There are circumstances under which the shellhooks are not effective.

You should prefer to set envvars via attributes whenever possible, as that is more reliavble.

1 Like
7

Thanks for the warning. How would I define the library path through an attribute?

1 Like
8

If you could point me in the direction where I should look in the documentation that would also be great.

9 
mkShell {
  LD_LIBRARY_PATH = "${stdenv.cc.cc.lib}/lib";
}
9 Likes
Sqlalchemy python fails to find libstdc++.so.6 in virtualenv
10

Thank you, this solved a longstanding problem for me. Just for completness and for everybody who comes after me (especially myself a few weeks in the future), here is a complete shell.nix that works for me (you probably don’t need the unstable part):

let
  unstable = import (fetchTarball "https://releases.nixos.org/nixos/unstable/nixos-22.11pre421761.\
448a599c499/nixexprs.tar.xz") { };
in
{ nixpkgs ? import <nixpkgs> {} }:

nixpkgs.mkShell {
  nativeBuildInputs = with nixpkgs; [
    unstable.python310Packages.poetry
    unstable.python310
  ];

  LD_LIBRARY_PATH = "${nixpkgs.stdenv.cc.cc.lib}/lib";
}
9 Likes
11

It took 3 months, but it was worth it.

4 Likes
12

This has worked to me:

  environment = {
    sessionVariables = {
      LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
    };
  };
13

No, please give What package provides libstdc++.so.6? - #5 by NobbZ a read.

1 Like
14

What’s the difference between this and your own suggestion?

15

That you put it into the global environment, rather than a dev shell.

16

For other readers, especially if you have multiple libraries you might consider lib.makeLibraryPath, which will combine its inputs properly and add the /lib for you. For example:

pkgs.mkShell {
    LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.stdenv.cc.cc ];
}
14 Likes
17

is there a way to check the versions going into pkgs.stdenv.cc.cc? like if I need libstdc++ to be >= 7.1`

18

Thanks, this works for me.

19

Might be a bit late to the party here but if anyone is struggling a bit to make this work still, I was able to resolve it using a shell.nix file in the repository. I am using uv to manage the project.

My configuration.nix looks like this:

  nixpkgs.config.allowUnfree = true;
  environment.systemPackages = with pkgs; [
    unixODBC
    unixODBCDrivers.msodbcsql18
    sqlcmd
  ];
  environment.unixODBCDrivers = with pkgs.unixODBCDrivers; [ msodbcsql18 ];
  programs.nix-ld.enable = true;

I then have the following shell.nix in the top folder of my python project:

with import <nixpkgs> { };
pkgs.mkShell {
  name = "loc-engine";

  # Here is where you will add all the libraries required by your native modules
  # You can use the following one-liner to find out which ones you need.
  # Just make sure you have `gcc` installed.
  # `find .venv/ -type f -name "*.so" | xargs ldd | grep "not found" | sort | uniq`
  NIX_LD_LIBRARY_PATH = lib.makeLibraryPath [
    stdenv.cc.cc # libstdc++
    stdenv.cc.cc.lib
    zlib # libz (for numpy)
    #libstdc++.so.6
    #libz.so.1
  ];

  NIX_LD = lib.fileContents "${stdenv.cc}/nix-support/dynamic-linker";
  LD_LIBRARY_PATH = "${stdenv.cc.cc.lib}/lib";

  packages = with pkgs; [
    uv
  ];

}

(I got this from crescentrose, which was the only way I could get things working)

Once in the root folder (and after a nix-os rebuild), run nix-shell (I use nix-shell –run zsh to preserve my zsh terminal) and you should then be able to do your usual uv sync functions from here).

A useful tip in this case where your collaborators may not be using nix is to put shell.nix in .git/info/exclude, which is like .gitignore but will not appear to the other collaborators in the repo

1 Like