How to prevent app from loading `/run/opengl-driver`?

Help
1

I have some software packaged against nixpkgs 21.11, using nix-build (which is supposed to run anywhere using nix, including Ubuntu, NixOS, and so on). It worked fine on my NixOS 21.11 laptop.

After upgrading the laptop to NixOS 22.05, my app (unmodified) now errors with:

myapp: /nix/store/s5zfi8x1r4h807qwbb2dxqw8jz7ipf20-glibc-2.33-59/lib/libc.so.6: version `GLIBC_2.34' not found (required by /run/opengl-driver/lib/libva.so.2)

It is trying to load my /run/opengl-driver/lib/libva.so.2, which was built against (and probably uses a function from) the newer glibc 2.34 that comes with NixOS 22.05. But my app is linked against the older glibc.so.6 from my pinned nixpkgs 21.11. So dynamic loading fails.

libva.so (video accelleration) is loaded because my app uses OpenCV and thus ffmpeg.

On my server that doesn’t have a GPU, myapp starts perfectly fine, and libva.so isn’t even loaded because it doesn’t exist, as strace shows:

openat(AT_FDCWD</root "/run/opengl-driver/lib/libva.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

It doesn’t exist because of this symlink on the server:

/run/opengl-driver/lib -> /nix/store/ybkk456fa7hwh52l0lirsjg89zsgyihq-mesa-21.2.5-drivers/lib

And that mesa derivation doesn’t ship libva.so.

But my laptop uses NVIDIA and thus has libva.so.

I do not wish my app to use native video acceleration. myapp just uses OpenCV for some things that don’t need this.

Can I, at runtime, convince my nix-built myapp binary to not load libva.so, so that it behaves just as on the server without GPU?

Ideally I would be able to use the same binary (given that it works on the GPU-less server) and not need to rebuild an OpenCV that uses no ffmpeg at all.

1 Like
2

see [RFC 0121] Migrate OpenGL References to API-Agnostic Terms by jonringer · Pull Request #121 · NixOS/rfcs · GitHub :slight_smile:

3

I don’t think that would help. That RFC is more about naming things correctly rather than changing any functionality. The issue in this case is that the driver wants a newer version of glibc than the application is providing.

2 Likes
4

Just to be super clear for my case: The issue is that the driver is used at all.

What I want is the equivalent of libva.so not being looked for or or found (like on systems where it doesn’t exist).

5

Failed try: libredirect.so

I tried with pkgs/build-support/libredirect/libredirect.c, which can rewrite libc path accesses at runtime, to try and hide /run/opengl-driver/lib, like so:

strace -fy --env=LD_PRELOAD=/nix/store/dgw12pb1hi6dfas5xzrpjqifyfqx5gm5-libredirect-0/lib/libredirect.so --env=NIX_REDIRECTS=/run/opengl-driver/lib/libva.so.2=/nonexistent myapp 2>&1 | grep -E 'opengl|nonexistent'

But to no avail, the rewrite doesn’t seem to kick in, /noexistent does not appear in the strace output.

Reason

On Matrix @Sandro, @tpw_rules and me found this:

Here the code

    # Set RUNPATH so that libnvcuvid and libcuda in /run/opengl-driver(-32)/lib can be found.
    # See the explanation in addOpenGLRunpath.
    addOpenGLRunpath $out/lib/libavcodec.so
    addOpenGLRunpath $out/lib/libavutil.so

adds a hardcoded lookup to /run/opengl-driver/lib into these .so files.

We can verify that using:

patchelf --print-rpath /nix/store/jy5x7c3svng38xr37pgqsjgmvq47yv2x-ffmpeg-4.4.1/lib/libavutil.so

which prints

/run/opengl-driver/lib:...

This RPATH entry is probably also why the libredirect.so approach didn’t work: RPATH loads seem to be not rewritable by it (not sure why, does the kernel interpret RPATH directly?).

Workaround (crude)

I’m having success with:

LD_LIBRARY_PATH=$HOME/tmp/changed-lib/ myapp

where $HOME/tmp/changed-lib/ contains

libavcodec.so
libavutil.so

from the above lines, but modified with patchelf --set-rpath to the value of patchelf --print-rpath on each, but with the /run/opengl-driver/lib entry removed.

1 Like
6

Correct, the RFC just wants hardware driver path to not be named after a graphics api.

7

I might be missing something, but I don’t think your app would even be trying to load libva.so unless the Nvidia driver had a dependency on it. And your app shouldn’t be loading the Nvidia driver unless it’s actually using APIs from it. Is that not correct? Are you sure your app would actually function if you managed to prevent it from loading libva.so? The fact that it works on your server without libva.so just means that mesa doesn’t have a hard dependency on it.

8

Related: Python package with runtime dependencies - #14 by jonringer