Unify glibc for bash and python and ghc

yaitskov · September 7, 2023, 3:37pm

Hi,

I noticed that glibc version used by bash and python and ghc might get out of sync. I preload python as a shared library and it breaks shell commands.

I think I need to customize bash derivation to use it with newer glibc version.

I didn’t touch glibc anywhere in my default.nix

ElvishJerricco · September 7, 2023, 3:43pm

nixpkgs only uses one glibc at a time. If you’ve got different packages using different glibc’s, it’s probably because you’ve gotten them from different revisions of nixpkgs.

yaitskov · September 29, 2023, 11:53pm

I prepared a simple demo project which demonstrates how easy to mix glibc versions. The script makes shell, imports stock python interpreter, preloads python library and launches sub shell. In my case ${pkgs.bash}/bin/bash works meanwhile default bash gets glibc conflict. For bash script it is not a big deal but some libraries spawn subprocesses and depend on system call. I think the best solution is to setup derivation for bash with custom glibc (stdenv) and use it for mkShell. Though I haven’t found an example.

github.com

yaitskov/nix-glibc-salad/blob/8432cd4cc2152860a4cdf1536c94735d1abd196c/default.nix

{ system ? builtins.currentSystem or "x86_64-linux" }:

let
  nix = import ./nix;
  pkgs = nix.pkgSetForSystem system {};

  myPython = pkgs.python311;

  shell = pkgs.mkShell {
    packages = [ myPython ];

    shellHook = ''
      LD_PRELOAD=${myPython.out}/lib/libpython3.so ${pkgs.bash}/bin/bash -c 'echo hello world'
      LD_PRELOAD=${myPython.out}/lib/libpython3.so bash -c 'echo hello world'
    '';
  };
in {
  inherit shell;
}

ElvishJerricco · September 30, 2023, 12:13am

Your example repo works for me. I just see hello world twice when I use nix-shell.

yaitskov · September 30, 2023, 12:19am

Wow - nonreproducable nix build

$ nix-shell --version
$ nix-shell (Nix) 2.11.0

$ nix-shell --pure
hello world
bash: /nix/store/8xk4yl1r3n6kbyn05qhan7nbag7npymx-glibc-2.35-224/lib/libc.so.6: version `GLIBC_ABI_DT_RELR' not found (required by /nix/store/ibp4camsx1mlllwzh32yyqcq2r2xsy1a-glibc-2.37-8/lib/libm.so.6)

ElvishJerricco · September 30, 2023, 12:20am

$ nix-shell --version
nix-shell (Nix) 2.17.0

ElvishJerricco · September 30, 2023, 12:36am

Even if I use nix 2.11.0 I still can’t reproduce this. Are you sure your nix store is healthy? nix-store --verify --check-contents

TLATER · September 30, 2023, 1:50am

Any chance your $PATH ends up ordered weirdly? Dumping env in general and sifting for anything weird might not be a bad idea. If my suspicions are correct, adding bash to packages would fix it.

This is a shellHook, not a nix build, which despite --pure does keep some things for usability reasons.

I believe there was some discussion around whether nix develop should start bash interactively like nix-shell because it’s more usable that way, which was ultimately rejected because otherwise exactly this can happen due to user .bashrcs. But I forgot the details of that issue, may even have been the exact opposite.