autoPatchelfHook not patching all dependencies

1

I’m getting some confusing autoPatchelfHook behavior. I have a derivation that pulls down a python wheel with binaries and then attempts to patch them with autoPatchelfHook. Without any autoPatchelfHook I get

> ldd result/lib/python3.9/site-packages/jaxlib/_pocketfft.so
        linux-vdso.so.1 (0x00007fff714fd000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ff031943000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff031928000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff031905000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff031713000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff031bb4000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff0315c4000)

Ok, makes sense lots of nix-unfriendly paths. Then if I add just nativeBuildInputs = [ autoPatchelfHook ]; I get an error:

autoPatchelfHook could not satisfy dependency libstdc++.so.6

which also makes sense to me, since I need to add those things as dependencies. So I add buildInputs = [ stdenv.cc.cc ];, and then everything builds just fine, but I see:

> ldd result/lib/python3.9/site-packages/jaxlib/_pocketfft.so
result/lib/python3.9/site-packages/jaxlib/_pocketfft.so: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/0kiykyrnrpfhmjwxwx89kxr20hmf5304-gcc-10.3.0-lib/lib/libstdc++.so.6)
        linux-vdso.so.1 (0x00007ffc2cb77000)
        libstdc++.so.6 => /nix/store/0kiykyrnrpfhmjwxwx89kxr20hmf5304-gcc-10.3.0-lib/lib/libstdc++.so.6 (0x00007fb9d707c000)
        libgcc_s.so.1 => /nix/store/0kiykyrnrpfhmjwxwx89kxr20hmf5304-gcc-10.3.0-lib/lib/libgcc_s.so.1 (0x00007fb9d7062000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb9d7026000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb9d6e34000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fb9d72ca000)
        libm.so.6 => /nix/store/9bh3986bpragfjmr32gay8p95k91q4gy-glibc-2.33-47/lib/libm.so.6 (0x00007fb9d6cf3000)

which still appears to contain a bunch of nix-unfriendly paths.

Why isn’t autoPatchelfHook touching these other dependencies? Is everything ok?

2

The docs seem to indicate that this shouldn’t happen:

By default autoPatchelf will fail as soon as any ELF file requires a dependency which cannot be resolved via the given build inputs.

3

What happens if you use ldd from nixpkgs? If you don’t have it in your nix environment/shell it is part of glibc-bin, i.e /nix/store/*glibc-*-bin/bin/ldd.

1 Like
4

Ah interesting… that seems to make quite a difference!

> nix-shell --pure -p glibc
[nix-shell:~/dev/nixpkgs]$ ldd result/lib/python3.9/site-packages/jaxlib/_pocketfft.so  
        linux-vdso.so.1 (0x00007fffc8eec000)
        libstdc++.so.6 => /nix/store/0kiykyrnrpfhmjwxwx89kxr20hmf5304-gcc-10.3.0-lib/lib/libstdc++.so.6 (0x00007f1e86661000)
        libgcc_s.so.1 => /nix/store/0kiykyrnrpfhmjwxwx89kxr20hmf5304-gcc-10.3.0-lib/lib/libgcc_s.so.1 (0x00007f1e86647000)
        libpthread.so.0 => /nix/store/9bh3986bpragfjmr32gay8p95k91q4gy-glibc-2.33-47/lib/libpthread.so.0 (0x00007f1e86627000)
        libc.so.6 => /nix/store/9bh3986bpragfjmr32gay8p95k91q4gy-glibc-2.33-47/lib/libc.so.6 (0x00007f1e86462000)
        /nix/store/9bh3986bpragfjmr32gay8p95k91q4gy-glibc-2.33-47/lib64/ld-linux-x86-64.so.2 (0x00007f1e868af000)
        libm.so.6 => /nix/store/9bh3986bpragfjmr32gay8p95k91q4gy-glibc-2.33-47/lib/libm.so.6 (0x00007f1e86321000)

If autoPatchelfHook completes successfully is it fair to assume that everything will be linked correctly at runtime then? This looks pretty good AFAICT.

5

Unfortunately not. In my case patchelf produced broken executables which didn’t run, but given that you have so few dependencies it may be ok. You may want to check the issues page Issues · NixOS/patchelf · GitHub.

1 Like
6

Sorry to jump here, but it’s one of the first result I get when googling error: auto-patchelf could not satisfy dependency libstdc++.so.6. In that case you want to add:

  buildInputs = [
    stdenv.cc.cc.lib
  ];
3 Likes
7

@tobiasBora thank you!