OpenGL and Vulkan should not be treated the same way as other libraries

Development
1

I’m not sure if this has been discussed before, but I am of the opinion that OpenGL and Vulkan should not be treated like libraries. On the surface, OpenGL/EGL and Vulkan are dynamically linked libraries. However, I would argue that they are actually protocols, that just happen to involve dynamically loading a library.

Now, just to be clear, I am tackling this from the perspective of someone using NixOS as a daily driver, I don’t have much experience with NixOS on the deployment end.

Nix does a really good job at taking dynamically linked libraries and allowing some apps to be updated to the latest version while keeping others on older versions for compatibility reasons. This is really nice as a developer and as a system manager so that I don’t have to worry about the “It works on my machine” problem (as much). However, this is not the case for EGL/Vulkan, and never will be.

EGL and Vulkan are not a library that you target. They are an interface to load the GPU drivers. Even if the version of EGL and Vulkan are the same across a set of computers, you have no guarantee that GPU drivers will be the same. Different drivers have different extensions and different bugs, and none of this is something that nix can hope to wrangle without forcing programs to target specific hardware. On the flipside, this is something that graphics programmers are well aware of and are prepared to work around.

What they can’t work around is EGL or Vulkan simply not being there.

You see, while most Linux programs use dynamically linked libraries, not all Linux programs do. I myself prefer to make statically linked programs, therefore guaranteeing that the code I ship doesn’t have it dependencies shifting underneath it, similar to the benefits Nix brings by hashing its inputs. Ideally, the programs I write should be able to run anywhere, because it is simply x86_64 machine code adhering to some stable interfaces: the syscall interface of the Linux kernel, the Wayland protocol, and the EGL/Vulkan libraries. As long as these interfaces keep their promises, my app should be able to run.

NixOS breaks the promises of the EGL and Vulkan interfaces. It hides these libraries from executables unless the user goes through extra effort to unhide them. While this may be desirable to prevent random libraries updates from breaking everything, EGL and Vulkan are standardized. And if you think that interfaces shouldn’t be exposed by default, you must explain why it is okay for Linux syscalls and Wayland unix domain sockets to be exposed by default, but not EGL and Vulkan. Just because EGL and Vulkan happens to use an ELF DLL doesn’t mean that they aren’t interfaces.

Anyway, that’s my rant. Do you agree or disagree? Do you think my hopes of releasing statically linked programs that run on most Linux distros are misguided or unrealistic? Or do you think I should find a different distro, since I don’t want to package everything into the one true packaging language, Nix?

P.S. not trying to be spicy but this has been simmering for a bit.

1 Like
2

/run/opengl-driver/lib and /run/opengl-driver-32/lib can be added to LD_LIBRARY_PATH (or DT_RUNPATH I guess) so this question/rant doesn’t make sense to me. Nixpkgs does so via addDriverRunpath or whatever it’s called now.

1 Like
3

Yea, this is literally what libglvnd and vulkan-loader do. You link directly to the loader library, and then the actual implementation is loaded dynamically at runtime; on NixOS it’s loaded from /run/opengl-driver

EDIT: I should note that currently you can’t use packages from nixpkgs that link to a different version of libgbm than the one used in your mesa drivers at runtime, which is very unfortunate because it means you can’t actually use random nixpkgs packages with random nixos systems. But a fix for this problem has been merged upstream and will hopefully make it into nixpkgs before too long.

5 Likes
4

Can you give an example of an application that’s harder to run on NixOS because of the fact EGL/Vulkan isn’t given special treatment?

5

/run/opengl-driver/lib and /run/opengl-driver-32/lib can be added to LD_LIBRARY_PATH (or DT_RUNPATH I guess) so this question/rant doesn’t make sense to me. Nixpkgs does so via addDriverRunpath or whatever it’s called now.

I don’t deny that you can fix it, and I have used nix-ld to achieve something something similar (although thinking on it the LD_LIBRARY_PATH is probably a better way to solve it). My issue is more to do with the default installation (again, commenting mostly about the desktop side, people running servers probably don’t care).

NixOS has an option to enable OpenGL. That’s good. The bad part is that enabling OpenGL doesn’t install it in a standardized way that would make it possible for an application to run on NixOS without specifically targeting NixOS. I argue that enabling OpenGL should automatically put it in LD_LIBRARY_PATH, or else it is only installed in the flimsiest sense of “I have it downloaded to my PC.”

And again, there are many interfaces which NixOS does not attempt to restrict in the same way, as far as I understand. Linux kernel syscalls are still available. I can open a connection to a wayland compositor and do software rendering all day. Both of these are installed properly as long as I’m using Linux and have a wayland compositor installed. But for some reason OpenGL and Vulkan are not installed properly unless you manually override it in multiple ways.

Can you give an example of an application that’s harder to run on NixOS because of the fact EGL/Vulkan isn’t given special treatment?

Yes, my own applications. I have NixOS configured to expose libEGL and Vulkan though nix-ld. However, just setting up nix-ld isn’t enough to make it work, I also need a block of code to load the NIX_LD_LIBRARY_PATH environment variable to make it work:

    if (std.process.getEnvVarOwned(arena, "NIX_LD_LIBRARY_PATH")) |path_list| {
        var path_list_iter = std.mem.tokenize(u8, path_list, ":");
        while (path_list_iter.next()) |path| {
            try prefixes_to_try.append(path);
        }
    } else |_| {}

If I comment out this block of code, my program is unable to find libvulkan.so:

warning: could not load "libvulkan.so", searched paths:
warning: 	.
warning: 	
warning: 	/usr/lib/
warning: 	/home/geemili/src/1_projects/seizer/.zig-cache/o/0e13f20781ff1a72eecc2507c5c31221/lib
warning: 	/home/geemili/src/1_projects/seizer/.zig-cache/o/lib
warning: 	/home/geemili/src/1_projects/seizer/.zig-cache/lib
warning: 	/home/geemili/src/1_projects/seizer/lib
warning: 	/home/geemili/src/1_projects/lib
warning: 	/home/geemili/src/lib
warning: 	/home/geemili/lib
warning: 	/home/lib
warning: 	/lib
warning(seizer): Failed to create Graphics.Driver.vulkan graphics context: error.LibraryLoadFailed

Now, I can probably fix this specific issue by setting LD_LIBRARY_PATH in my NixOS config instead of NIX_LD_LIBRARY_PATH. But perhaps I should ask a more specific question:

Why aren’t OpenGL and Vulkan automatically added to LD_LIBRARY_PATH when they are enabled? Does the Nix project gain any benefit from not doing this by default, that is worth the extra incompatibility it causes?

Again, sorry if I sound combative. I’m just frustrated by how difficult it is to target Linux as a platform, and I want to see it improve. TBH I don’t even know if what I’m saying would be a step in the right direction, or even matter at all, but since I daily drive NixOS it’s at the top of my mind.

1 Like
6

I got my program to work with nix-ld, it was a compiler issue. zig was preferring a specific glibc dynamic linker (/nix/store/r8qsxm85rlxzdac7988psm7gimg4dl3q-glibc-2.39-52/lib/ld-linux-x86-64.so.2) instead of the more generic /lib64/ld-linux-x86-64.so.2.

I don’t know that this invalidates my previous points, but it makes me less certain about whether this was a nix issue or a zig issue.