Software rendering using mesa not possible?

Hi, I’d like to use mesa (e.g. mesa-noglu-18.3.4) for software rendering, because the GUI I’m packaging doesn’t depend on high-performance graphics, and software rendering would be a nice way to make the package more pure.

Somewhat like this, but replace glxinfo with the app:

$ LIBGL_ALWAYS_SOFTWARE=1 GALLIUM_DRIVER=llvmpipe glxinfo

But when I use mesa, I always see libglvnd sending requests to my X server, presumably to ask which vendor to use.

$ nix-shell -p mesa glxinfo
[nix-shell:~]$ LIBGL_ALWAYS_SOFTWARE=1 LIBGL_DEBUG=verbose strace glxinfo
...
socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC, 0) = 3
connect(3, {sa_family=AF_LOCAL, sun_path=@"/tmp/.X11-unix/X0"}, 20) = 0
...
openat(AT_FDCWD, "/nix/store/69vq0a9sqynmz335apm8zgyjdmq34s5j-libX11-1.6.7/lib/libGLX_nvidia.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/lswf09qbkkrqd0rgzaqyrkr44lf78y9x-libXext-1.3.4/lib/libGLX_nvidia.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/nj5w33n15zrn8xsm9q0vf3pk0j0j9sj8-libglvnd-1.0.0/lib/libGLX_nvidia.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
...

In my case I have an nvidia driver installed on Ubuntu 16.04 — non NixOS. So it looks for libGLX_nvidia instead of libGLX_mesa. To be honest, I’m not 100% sure if libGLX_mesa is necessary for software rendering, but libGLX_nvidia is definitely the wrong thing.

Thoughts? Would a simple mesa libGL.so (instead of going through libglvnd) be correct? Curious why mesa-noglu-18.3.4 doesn’t have one.

To find those specific packages, you will want the nvidia specific drivers instead of the generic mesa ones. I think the command you’re looking for is:

nix-shell -p linuxPackages.nvidia_x11

on my machine:

$ ls /nix/store/sh8qyg8b3f0hgh6v36l19a54w0h2djnk-nvidia-x11-418.74-4.19.44/lib/ | grep libGLX
libGLX_nvidia.so -> libGLX_nvidia.so.418.74
libGLX_nvidia.so.0 -> libGLX_nvidia.so.418.74
libGLX_nvidia.so.418.74

You may need to do allowUnfree = true for nvidia packages

As to the issue of which dll is the most correct, ¯\_(ツ)_/¯ i dont know

@jonringer Thanks! But in this case, I’d like to package a version of libGL.so that implements software rendering with the application. The main reason is so a new user doesn’t need to lookup their graphics driver, nvidia version, etc. It’s not a very high-performance app and it’s used only for demo purposes. Intended instructions:

$ curl https://nixos.org/nix/install | sh
$ nix-shell -p app
[nix-shell:~]$ app

BTW - Check out nixGL, which is pretty cool. You need to know your nvidia graphics card details, but it pulls down the right version of the nvidia drivers for your setup. Unfortunately this is still too advance (and failure prone) for the users of the app I’m packaging.

Oh, you’re going to redistribute this on an unknown, platform. In that case, this is also out of my expertise, sorry :frowning:

I think the nixos-install also avoids directly involving itself with determining video drivers, so i have no idea.

Only thing that comes to mind is doing some “impure” wrapper to determine it like glxinfo -B and scraping the text for certain things, but this seems error prone and fragile.

Wish you the best of luck :smiley:

the tensorflowWithCuda package just checks to see if nvidia_x11 is populated, you may be able to inspect if the package exists in their pkgs.

tensorflow =
  if stdenv.isDarwin
  then callPackage ../development/python-modules/tensorflow/bin.nix { }
  else callPackage ../development/python-modules/tensorflow/bin.nix rec {
    cudaSupport = pkgs.config.cudaSupport or false;
    inherit (pkgs.linuxPackages) nvidia_x11;
    cudatoolkit = pkgs.cudatoolkit_10_0;
    cudnn = pkgs.cudnn_cudatoolkit_10_0;
  };

tensorflowWithoutCuda = self.tensorflow.override {
  cudaSupport = false;
};

tensorflowWithCuda = self.tensorflow.override {
  cudaSupport = true;
};

I’m not sure how this will work or not work for your ubuntu use case, but maybe it will :slight_smile:

This means that you would have to make a derivation within the script, evaluate the derivation, and execute the build product, but it is possible. I think :upside_down_face:

I think you probably want to link against libGLX_mesa. These are hidden in the “drivers” output of mesa, and normally are only used by libglvnd. Something like this should work:

-L${mesa.drivers}/lib -lGLX_mesa

These are hidden because we normally don’t want apps to link against this, and use the system provided libGL. I think your use case it makes sense, but this is definitely hidden. You will need to make a stub pkgconfig file as well in $out/lib/pkgconfig/libGL.pc:

Name: libGL
Description: Software-based libGL library
Version: 0.0
Libs: -L${mesa.drivers}/lib -lGLX_mesa
Cflags: -I${mesa.dev}/include

It might be useful to include this in nixpkgs as something like libGL, but we want to highly discourage this for most packages.

This might be more difficult if you are using a prebuilt binary. You can probably use something like LD_PRELOAD for this though. Otherwise might need a symlink, but this sounds a little bit dangerous.

@matthewbauer Thanks for the tip! Is that equivalent to setting __GLX_VENDOR_LIBRARY_NAME=mesa at runtime? I reached out to the libglvnd folks (see thread here) and that’s what they recommended.

I’m also curious, why is it highly discouraged for packages to not depend on system libs? For non-NixOS users, depending on a system libGL rarely works out of the box and has been a big pain point (I myself have spent the last 2 weeks trying to make this easier for non-NixOS users). It seems like giving users the option to depend on a simple libGL built only for software rendering would be an improvement.

Yeah that looks like it! The only difference is __GLX_VENDOR_LIBRARY_NAME will still use libglvnd and not require recompiling anything. That definitely looks like a better solution.

It definitely depends on what you are using GL for. But, for heavy duty usage, you definitely want a driver that utilizes your GPU. This is going to be different on every computer, so it needs to be provided by the system. Doing one of the above (either __GLX_VENDOR_LIBRARY_NAME or linking to libGLX_mesa) will override that auto detection.

But the experience right now is definitely not great. I think we can improve this by maybe falling back to __GLX_VENDOR_LIBRARY_NAME if no GL is found. In addition we could probably add /usr/lib to the RPATH like was done in https://github.com/NixOS/nixpkgs/pull/60985.

Opened https://github.com/NixOS/nixpkgs/issues/62169 to track this discussion & try to get feedback from Vlad who originally authored this stuff.