What is the best way to override the nvidia driver to create the symbolic link from libnvidia-allocator.so to nvidia-drm_gbm.so? What I’m doing now is:
environment.etc."gbm/nvidia-drm_gbm.so".source = "${nvidia_beta}/lib/libnvidia-allocator.so";
Then setting GBM_BACKENDS_PATH to /etc/gbm. It’s working, but I think it’s ugly.
I have commits for that on my nixpkgs. We should fix it in nixpkgs, feel free to +1 my PR for “fix egl-wayland” where I’ve added the fix for this too.
Note, you’ll need my mesa PR too which will cause mass rebuilds.
Ahh, this was just with glxgears/glxinfo haha. There are some quirks that lead me to believe that it’s not completely working right, waiting on the Mesa PR to make it into staging or master before testing further.
I’m trying to figure out how to follow along from home using 495 without eglstreams. I’ve got a vanilla install on nixos-unstable with just nvidia.nix and wayland-tweaks.nix copied over, but that doesn’t seem to be enough. (I’ve also used the GBM_BACKENDS_PATH trick with /etc above).
The default sway session doesn’t start (normal, it doesn’t have any of the fixups mentioned here), but starting nvidia-sway results in a black screen with nothing. /tmp/sway.log shows everything working
Here’s the sway.log:
EDIT: I just noticed this line in the log:
00:00:00.041 [INFO] [wlr] [backend/headless/backend.c:227] Creating headless backend with parent renderer
Further EDIT:
I think I have my answer. Using nixpgs-wayland is required in order for this to work as well. I’ll add that as an overlay and try again
After adding a nixpgs-wayland overlay, I have sway up and running, but without GPU acceleration. Current paste:
Trying steam on wayland with nvidia:
Loaded SDL version 2.0.17-6744061
OpenGL GLX extension not supported by display
OpenGL GLX extension not supported by displaysrc/steamUI/Main.cpp (409) : Assertion Failed: Fatal Error: OpenGL GLX extension not supported by display
src/steamUI/Main.cpp (409) : Assertion Failed: Fatal Error: OpenGL GLX extension not supported by display
Installing breakpad exception handler for appid(steam)/version(1634158817)
DBG DBG ffffffff DUMP_REQUESTED Assert( Assertion Failed: Fatal Error: OpenGL GLX extension not supported by display ):/data/src/steamUI/Main.cpp:409
Though that is what I have always gotten over wayland before the 2 PRs/4 commits.
mesa: add support for locating gdm's through /run/opengl-driver/lib
nvidia: install egl wayland gbm support
nvidia: add wayland to nvidia-egl-wayland libpath
nvidia: fix egl-wayland loading
I think I might be missing the EGL overlay for steam, investigating. However I can at least report everything that worked before the patches is still very functional after a whole day of usage.
Hi @colemickens. Is there any problem blocking the merge of this PR nvidia: fix egl-wayland loading by colemickens · Pull Request #139354 · NixOS/nixpkgs · GitHub ?
I finally got everything working after a chat with a colleague helping me to understand more about how things work, and then reading through the nix pills again (this time with enough understanding to follow everything). I’m using an overlay to replace a couple of packages with those coming from @colemickens but while sway works just find, as does accelerated firefox, I can’t get xwayland to work accelerated. When I run eglinfo, I get X11 platform: eglinfo: eglInitialize failed
As I understand it [staging] mesa: include /run/opengl-driver/lib in gbm path by jonringer · Pull Request #141768 · NixOS/nixpkgs · GitHub or some “replacement” is also necessary.
The mesa change is in the nixpkgs fork of @colemickens . I’m using this right now, and it’s better than the alternative, but I still can’t yet build steam (the overlay mixing nixpkgs unstable with the custom packages has some weird issues I’ve had to work around). I’ve got 3D acceleration working for things like glmark, glxgears, and firefox, so at least there’s that. Unfortunately, I’m unable to get my monitor to full resolution because the nvidia driver doesn’t seem to support display stream compression (EDID reports available modes that get filtered out by the kernel DRM). I may just try out i3 on X to see if things are better there.
For anyone who wants to follow at home, I wanted to document the simplest config that gets you to where I am. You need a local copy of the builder.sh from the nvidia_x11 package, because it’s been modified, in my case it’s next to my config and called nvidia_x11_builder.sh
The following is in my configuration.nix:
{ lib, config, pkgs, ... }:
let
nvidia-acceleration-overlay = (self: super: {
linuxPackages = super.linuxPackages.extend (final: prev: {
nvidia_x11.args = [ "-e" ./nvidia_x11_builder.sh ];
nvidia_x11.libPath = super.pkgs.lib.makeLibraryPath [ super.pkgs.libdrm super.pkgs.xorg.libXext super.pkgs.xorg.libX11 super.pkgs.xorg.libXv super.pkgs.xorg.libXrandr super.pkgs.xorg.libxcb super.pkgs.zlib super.pkgs.stdenv.cc.cc super.pkgs.wayland super.pkgs.libglvnd ];
nvidia_x11.libPath32 = super.pkgsi686Linux.lib.makeLibraryPath [ super.pkgsi686Linux.libdrm super.pkgsi686Linux.xorg.libXext super.pkgsi686Linux.xorg.libX11 super.pkgsi686Linux.xorg.libXv super.pkgsi686Linux.xorg.libXrandr super.pkgsi686Linux.xorg.libxcb super.pkgsi686Linux.zlib super.pkgsi686Linux.stdenv.cc.cc super.pkgsi686Linux.wayland super.pkgsi686Linux.libglvnd ];
});
mesa = super.mesa.overrideAttrs ( old: rec {
mesonFlags = super.mesa.mesonFlags ++ [ "-Dgbm-backends-path=/run/opengl-driver/lib/gbm:${placeholder "out"}/lib/gbm:${placeholder "out"}/lib" ];
});
wlroots = prev.wlroots.overrideAttrs(old: {
postPatch = ''
sed -i 's/assert(argb8888 &&/assert(true || argb8888 ||/g' 'render/wlr_renderer.c'
'';
});
xwayland = super.xwayland.overrideAttrs (old: rec {
version = "21.1.3";
src = super.fetchFromGitLab {
domain = "gitlab.freedesktop.org";
owner = "xorg";
repo = "xserver";
rev = "21e3dc3b5a576d38b549716bda0a6b34612e1f1f";
sha256 = "sha256-i2jQY1I9JupbzqSn1VA5JDPi01nVA6m8FwVQ3ezIbnQ=";
};
});
});
in
{
imports = [ ./nvidia.nix ./sway.nix ];
nixpkgs.overlays = [ nvidia-acceleration-overlay ];
}
Then you also need to following in a file called nvidia.nix:
{ config, pkgs, lib, ... }:
let
nvidiaPackage = config.boot.kernelPackages.nvidiaPackages.stable;
in
{
config = {
environment.etc."gbm/nvidia-drm_gbm.so".source = "${nvidiaPackage}/lib/libnvidia-allocator.so";
environment.etc."egl/egl_external_platform.d".source = "/run/opengl-driver/share/egl/egl_external_platform.d/";
hardware.opengl.enable = true;
hardware.opengl.driSupport = true;
hardware.opengl.driSupport32Bit = true;
hardware.pulseaudio.support32Bit = true;
hardware.opengl.extraPackages = with pkgs; [
vaapiVdpau
libvdpau-va-gl
libva
];
environment.systemPackages = with pkgs; [
firefox
vulkan-tools
libva-utils
];
hardware.nvidia.modesetting.enable = true;
hardware.nvidia.package = nvidiaPackage;
hardware.nvidia.powerManagement.enable = false;
services.xserver = {
videoDrivers = [ "nvidia" ];
displayManager.gdm.wayland = true;
displayManager.gdm.nvidiaWayland = true;
};
};
}
…and your sway config in sway.nix along with the following:
{ config, pkgs, lib, ... }:
{
programs.sway = {
enable = true;
extraOptions = [
"--unsupported-gpu"
"--my-next-gpu-wont-be-nvidia"
];
extraSessionCommands = ''
export MOZ_ENABLE_WAYLAND=1
export QT_QPA_PLATFORM=wayland
export QT_WAYLAND_DISABLE_WINDOWDECORATION="1"
export SDL_VIDEODRIVER=wayland
export XDG_CURRENT_DESKTOP="sway"
export XDG_SESSION_TYPE="wayland"
export _JAVA_AWT_WM_NONREPARENTING=1
export GBM_BACKEND=nvidia-drm
export GBM_BACKENDS_PATH=/etc/gbm
export __GLX_VENDOR_LIBRARY_NAME=nvidia
export WLR_NO_HARDWARE_CURSORS=1
'';
};
}
This doesn’t get you to steam, but it does get you acceleration working in wayland and sway.
I now have Steam working under Sway with NVIDIA. It seems that another patch to wlroots is necessary. Details here: https://gitlab.freedesktop.org/wlroots/wlroots/-/merge_requests/3338
(I’ve also bumped libglvnd/libdrm, not sure if necessary yet - this causes a huge rebuild).
Wunderbar! Great timing! I was just reading through the wlroots code side by side between 0.14.1 and master, and trying to figure out what the problem was. It was easy enough to see that display binding was removed and that device_enum wasn’t working, but it was looking to me like there was supposed to be an additional case for using GBM directly without device_enum. This solves the problem for me, so thank you!
I don’t use Sway, but I have Gnome-shell and Steam working in Wayland. My configs https://github.com/tfmoraes/nixoscfg/blob/e419481bbd1003a6d9bc96918d3308047ec6225d/host/watchmen/nvidia.nix
I tried it out, and it builds, with eglinfo showing support on wayland, and both glmark2 and glxgears working, however firefox is no longer accelerated (error about an inability to find a gpu on the pci bus), and chrome won’t even start. eglinfo still doesn’t show support on Xwayland, which is what I’ve been trying to fix.
I’ve reverted to the release version of wlroots, as I have more things working, though not steam.
I’ve now been at this for two weeks. After realizing that there is no support for hardware acceleration for video decoding in wayland, I abandoned this config and tried to go back to X.org with i3.
This made me realize that X.org is a mess, too, and not one that I think is worth it for the video acceleration.
Where I’m stuck right now is that every time I try to add steam, nixos-rebuild trys to rebuild a multi-architecture mesa, and the meson build system fails because it can’t detect the type of the compiler
I can’t really say I know anything about that, but I certainly know about going back to X11 and realizing that despite the warts it’s not worth looking backward.
What I do know, is that once nvidia_x11: libPath: add libgbm/libGL libraries by colemickens · Pull Request #145439 · NixOS/nixpkgs · GitHub is signed off and merged, it will just be a matter of waiting. (assuming use of nixpkgs-wayland). Everything that I’ve identified is submitted. I still don’t know why eglinfo fails in the X11 case, and I’ve opened a thread on the NVidia dev forums about the outstanding wlroots assert that everyone seems to patch around.
With the following 3 PRs steam runs great on nvidia-wayland for me:
commit b8a66217675f904b394448c1168800fe6fdb3df7 (HEAD -> staging-next)
Author: Cole Mickens <cole.mickens@gmail.com>
Date: Sat Nov 13 16:57:57 2021 -0800
nvidia_x11: style refactor: use "with pkgs"
commit 438178605a2cabc9f569b3eeb4c15d297f03033f
Author: Cole Mickens <cole.mickens@gmail.com>
Date: Sat Nov 6 17:24:23 2021 -0700
nvidia_x11: libPath: add libgbm/libGL libraries
commit be380735ab485789d8113b4c9bf5c7d73527b46c
Author: Cole Mickens <cole.mickens@gmail.com>
Date: Sat Oct 16 00:28:10 2021 -0700
mesa: /run/opengl-driver in gbm-backend-paths
commit e37830fc502c2748e5ce1f5048697cfda90a6ed0
Author: Abdelhakim Qbaich <abdelhakim@qbaich.com>
Date: Thu Sep 16 12:57:40 2021 -0700
egl-wayland: 1.1.7 -> 1.1.9
thanks to Nvidia Users Testers Requested (Sway on Nvidia, Steam on Wayland) - #32 by douglas I managed to get sway working with my nvidia GeForce GTX 970. There is some subtlte flickering though sometimes every 10 sec (I am in HDMI). Firefox or chrome decorations appear (including firefox tabs) but inside the window, I can’t do shit. I haven’t tried steam yet.