Proper setup for python development with Nix and VS Code

Help
1

Hi guys,

For a long time, my setup for python projects consisted of direnv, nix-shell for python (example below), requirements.txt from pip-tools, VS Code (mostly remote access), and recently, I added Nix Environment Selector for VSC (repository). But with this, I encountered a problem.

As some of python packages require libstdc++.so.6, I added LD_LIBRARARY_PATH with it to my shell hook (NixOS Wiki link), but once I’m trying to use it with VSC, all extensions start to crash with errors like: /home/.../.vscode-server/bin/.../node: /lib/x86_64-linux-gnu/libc.so.6: version 'GLIBC_2.34' not found (required by /nix/store/...-gcc-11.2.0-lib/lib/libgcc_s.so.1). To solve this, I only need to remove the LD_LIBRARARY_PATH from my shell hook, but then, I’m back to the problem with python packages (related issue).

Have you encountered something like this? Do you have any idea how to solve it? Or maybe, do you have an entirely different setup for python projects? Thank you!

shell.nix:

{ sources ? import ./nix/sources.nix }:

with import sources.nixpkgs {};

mkShell {
  name = "example-env";
  buildInputs = [
    python39
    nodejs
    glibcLocales
  ];
  shellHook = ''
  # set SOURCE_DATE_EPOCH so that we can use python wheels
  export SOURCE_DATE_EPOCH=315532800
  export LD_LIBRARY_PATH=${lib.makeLibraryPath [stdenv.cc.cc]}:$LD_LIBRARY_PATH
  '';
  preferLocalBuild = true;
}

.envrc:

use nix

VIRTUAL_ENV="$PWD/venv"
if [ ! -e venv ]
then
    python -m venv "${VIRTUAL_ENV}"
fi
export VIRTUAL_ENV
PATH_add "$VIRTUAL_ENV/bin"
3 Likes
2

Either package everything with Nix patching all paths to all libraries, or run everything in the same FHS environment. One possible work-around is to wrap the Python wrapper with your LD_LIBRARY_PATH.

1 Like
3

Thank you for the response! I tried the configurations you suggested and several others, and finally, I ended up with something like this (code below). It’s using autoPatchelf to change paths in all executables in venv, so I’ll end up with a “standalone” environment. So far, everything seems to work correctly, with the disadvantage that I need to rerun autoPatchelf with every requirements update.

shell.nix:

{ sources ? import ./nix/sources.nix }:

with import sources.nixpkgs {};

mkShell {
  name = "example-env";
  buildInputs = [
    python39
    python39Packages.venvShellHook
    autoPatchelfHook
  ];
  propagatedBuildInputs = [
    stdenv.cc.cc.lib
  ];

  venvDir = "./venv";
  postVenvCreation = ''
    unset SOURCE_DATE_EPOCH
    pip install -U pip setuptools wheel
    pip install -r requirements.txt -r requirements-dev.txt
    pip install -e .
    autoPatchelf ./venv
  '';

  # LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib:$LD_LIBRARY_PATH";

  postShellHook = ''
  # set SOURCE_DATE_EPOCH so that we can use python wheels
  export SOURCE_DATE_EPOCH=315532800
  unset LD_LIBRARY_PATH
  '';
  preferLocalBuild = true;
}

.envrc:

use nix

VIRTUAL_ENV="$PWD/venv"
export VIRTUAL_ENV
PATH_add "$VIRTUAL_ENV/bin"

Sources: