Nvidia Users Testers Requested (Sway on Nvidia, Steam on Wayland)

27

Trying steam on wayland with nvidia:

Loaded SDL version 2.0.17-6744061
OpenGL GLX extension not supported by display
OpenGL GLX extension not supported by displaysrc/steamUI/Main.cpp (409) : Assertion Failed: Fatal Error: OpenGL GLX extension not supported by display
src/steamUI/Main.cpp (409) : Assertion Failed: Fatal Error: OpenGL GLX extension not supported by display
Installing breakpad exception handler for appid(steam)/version(1634158817)
DBG DBG ffffffff DUMP_REQUESTED Assert( Assertion Failed: Fatal Error: OpenGL GLX extension not supported by display ):/data/src/steamUI/Main.cpp:409

Though that is what I have always gotten over wayland before the 2 PRs/4 commits.

mesa: add support for locating gdm's through /run/opengl-driver/lib
nvidia: install egl wayland gbm support
nvidia: add wayland to nvidia-egl-wayland libpath
nvidia: fix egl-wayland loading

I think I might be missing the EGL overlay for steam, investigating. However I can at least report everything that worked before the patches is still very functional after a whole day of usage.

28

Hi @colemickens. Is there any problem blocking the merge of this PR nvidia: fix egl-wayland loading by colemickens · Pull Request #139354 · NixOS/nixpkgs · GitHub ?

29

I finally got everything working after a chat with a colleague helping me to understand more about how things work, and then reading through the nix pills again (this time with enough understanding to follow everything). I’m using an overlay to replace a couple of packages with those coming from @colemickens but while sway works just find, as does accelerated firefox, I can’t get xwayland to work accelerated. When I run eglinfo, I get X11 platform: eglinfo: eglInitialize failed

30

As I understand it [staging] mesa: include /run/opengl-driver/lib in gbm path by jonringer · Pull Request #141768 · NixOS/nixpkgs · GitHub or some “replacement” is also necessary.

1 Like
31

The mesa change is in the nixpkgs fork of @colemickens . I’m using this right now, and it’s better than the alternative, but I still can’t yet build steam (the overlay mixing nixpkgs unstable with the custom packages has some weird issues I’ve had to work around). I’ve got 3D acceleration working for things like glmark, glxgears, and firefox, so at least there’s that. Unfortunately, I’m unable to get my monitor to full resolution because the nvidia driver doesn’t seem to support display stream compression (EDID reports available modes that get filtered out by the kernel DRM). I may just try out i3 on X to see if things are better there.

32

For anyone who wants to follow at home, I wanted to document the simplest config that gets you to where I am. You need a local copy of the builder.sh from the nvidia_x11 package, because it’s been modified, in my case it’s next to my config and called nvidia_x11_builder.sh
The following is in my configuration.nix:

{ lib, config, pkgs, ... }:
let
  nvidia-acceleration-overlay = (self: super: {
    linuxPackages = super.linuxPackages.extend (final: prev: {
      nvidia_x11.args = [ "-e" ./nvidia_x11_builder.sh ];
      nvidia_x11.libPath = super.pkgs.lib.makeLibraryPath [ super.pkgs.libdrm super.pkgs.xorg.libXext super.pkgs.xorg.libX11 super.pkgs.xorg.libXv super.pkgs.xorg.libXrandr super.pkgs.xorg.libxcb super.pkgs.zlib super.pkgs.stdenv.cc.cc super.pkgs.wayland super.pkgs.libglvnd ];
      nvidia_x11.libPath32 = super.pkgsi686Linux.lib.makeLibraryPath [ super.pkgsi686Linux.libdrm super.pkgsi686Linux.xorg.libXext super.pkgsi686Linux.xorg.libX11 super.pkgsi686Linux.xorg.libXv super.pkgsi686Linux.xorg.libXrandr super.pkgsi686Linux.xorg.libxcb super.pkgsi686Linux.zlib super.pkgsi686Linux.stdenv.cc.cc super.pkgsi686Linux.wayland super.pkgsi686Linux.libglvnd ];
    });
    mesa = super.mesa.overrideAttrs ( old: rec {
      mesonFlags = super.mesa.mesonFlags ++ [ "-Dgbm-backends-path=/run/opengl-driver/lib/gbm:${placeholder "out"}/lib/gbm:${placeholder "out"}/lib" ];
    });
    wlroots = prev.wlroots.overrideAttrs(old: {
      postPatch = ''
        sed -i 's/assert(argb8888 &&/assert(true || argb8888 ||/g' 'render/wlr_renderer.c'
      '';  
    });
    xwayland = super.xwayland.overrideAttrs (old: rec {
      version = "21.1.3";
      src = super.fetchFromGitLab {
        domain = "gitlab.freedesktop.org";
        owner = "xorg";
        repo = "xserver";
        rev = "21e3dc3b5a576d38b549716bda0a6b34612e1f1f";
        sha256 = "sha256-i2jQY1I9JupbzqSn1VA5JDPi01nVA6m8FwVQ3ezIbnQ=";
      };
    });  
  }); 
in
{
  imports = [ ./nvidia.nix ./sway.nix ];
  nixpkgs.overlays = [ nvidia-acceleration-overlay ];
}

Then you also need to following in a file called nvidia.nix:

{ config, pkgs, lib, ... }:
let
  nvidiaPackage = config.boot.kernelPackages.nvidiaPackages.stable;
in
{

  config = {
    environment.etc."gbm/nvidia-drm_gbm.so".source = "${nvidiaPackage}/lib/libnvidia-allocator.so";
    environment.etc."egl/egl_external_platform.d".source = "/run/opengl-driver/share/egl/egl_external_platform.d/";
    hardware.opengl.enable = true;
    hardware.opengl.driSupport = true;
    hardware.opengl.driSupport32Bit = true;
    hardware.pulseaudio.support32Bit = true;
    hardware.opengl.extraPackages = with pkgs; [
      vaapiVdpau
      libvdpau-va-gl
      libva
    ];
    environment.systemPackages = with pkgs; [
      firefox
      vulkan-tools
      libva-utils
    ];                         

    hardware.nvidia.modesetting.enable = true;
    hardware.nvidia.package = nvidiaPackage;
    hardware.nvidia.powerManagement.enable = false;
  
    services.xserver = {
      videoDrivers = [ "nvidia" ];
      displayManager.gdm.wayland = true;
      displayManager.gdm.nvidiaWayland = true;
    };
  };                     
}

…and your sway config in sway.nix along with the following:

{ config, pkgs, lib, ... }:
{
  programs.sway = {
    enable = true;
    extraOptions = [
      "--unsupported-gpu"
      "--my-next-gpu-wont-be-nvidia"
    ];
    extraSessionCommands = ''
        export MOZ_ENABLE_WAYLAND=1
        export QT_QPA_PLATFORM=wayland
        export QT_WAYLAND_DISABLE_WINDOWDECORATION="1"
        export SDL_VIDEODRIVER=wayland
        export XDG_CURRENT_DESKTOP="sway"
        export XDG_SESSION_TYPE="wayland"
        export _JAVA_AWT_WM_NONREPARENTING=1
        export GBM_BACKEND=nvidia-drm
        export GBM_BACKENDS_PATH=/etc/gbm
        export __GLX_VENDOR_LIBRARY_NAME=nvidia
        export WLR_NO_HARDWARE_CURSORS=1
      '';
    };
}

This doesn’t get you to steam, but it does get you acceleration working in wayland and sway.

4 Likes
33

I now have Steam working under Sway with NVIDIA. It seems that another patch to wlroots is necessary. Details here: https://gitlab.freedesktop.org/wlroots/wlroots/-/merge_requests/3338

(I’ve also bumped libglvnd/libdrm, not sure if necessary yet - this causes a huge rebuild).

34

Wunderbar! Great timing! I was just reading through the wlroots code side by side between 0.14.1 and master, and trying to figure out what the problem was. It was easy enough to see that display binding was removed and that device_enum wasn’t working, but it was looking to me like there was supposed to be an additional case for using GBM directly without device_enum. This solves the problem for me, so thank you!

35

I don’t use Sway, but I have Gnome-shell and Steam working in Wayland. My configs https://github.com/tfmoraes/nixoscfg/blob/e419481bbd1003a6d9bc96918d3308047ec6225d/host/watchmen/nvidia.nix

1 Like
36

I tried it out, and it builds, with eglinfo showing support on wayland, and both glmark2 and glxgears working, however firefox is no longer accelerated (error about an inability to find a gpu on the pci bus), and chrome won’t even start. eglinfo still doesn’t show support on Xwayland, which is what I’ve been trying to fix.

I’ve reverted to the release version of wlroots, as I have more things working, though not steam.

37

I’ve now been at this for two weeks. After realizing that there is no support for hardware acceleration for video decoding in wayland, I abandoned this config and tried to go back to X.org with i3.

This made me realize that X.org is a mess, too, and not one that I think is worth it for the video acceleration.

Where I’m stuck right now is that every time I try to add steam, nixos-rebuild trys to rebuild a multi-architecture mesa, and the meson build system fails because it can’t detect the type of the compiler

1 Like
38

I can’t really say I know anything about that, but I certainly know about going back to X11 and realizing that despite the warts it’s not worth looking backward.

What I do know, is that once nvidia_x11: libPath: add libgbm/libGL libraries by colemickens · Pull Request #145439 · NixOS/nixpkgs · GitHub is signed off and merged, it will just be a matter of waiting. (assuming use of nixpkgs-wayland). Everything that I’ve identified is submitted. I still don’t know why eglinfo fails in the X11 case, and I’ve opened a thread on the NVidia dev forums about the outstanding wlroots assert that everyone seems to patch around.

1 Like
39

With the following 3 PRs steam runs great on nvidia-wayland for me:

commit b8a66217675f904b394448c1168800fe6fdb3df7 (HEAD -> staging-next)
Author: Cole Mickens <cole.mickens@gmail.com>
Date:   Sat Nov 13 16:57:57 2021 -0800

    nvidia_x11: style refactor: use "with pkgs"

commit 438178605a2cabc9f569b3eeb4c15d297f03033f
Author: Cole Mickens <cole.mickens@gmail.com>
Date:   Sat Nov 6 17:24:23 2021 -0700

    nvidia_x11: libPath: add libgbm/libGL libraries

commit be380735ab485789d8113b4c9bf5c7d73527b46c
Author: Cole Mickens <cole.mickens@gmail.com>
Date:   Sat Oct 16 00:28:10 2021 -0700

    mesa: /run/opengl-driver in gbm-backend-paths

commit e37830fc502c2748e5ce1f5048697cfda90a6ed0
Author: Abdelhakim Qbaich <abdelhakim@qbaich.com>
Date:   Thu Sep 16 12:57:40 2021 -0700

    egl-wayland: 1.1.7 -> 1.1.9
1 Like
40

thanks to Nvidia Users Testers Requested (Sway on Nvidia, Steam on Wayland) - #32 by douglas I managed to get sway working with my nvidia GeForce GTX 970. There is some subtlte flickering though sometimes every 10 sec (I am in HDMI). Firefox or chrome decorations appear (including firefox tabs) but inside the window, I can’t do shit. I haven’t tried steam yet.

41

I’ve been meaning to comment, Xwayland acceleration still isn’t quite right (and I’m out of things to check).

Other than that, WLR_NO_HARDWARE_CURSORS and the argb8888/xrgb8888 patch, everything else is good to go in nixos-unstable.

1 Like
42

There is some subtlte flickering though sometimes every 10 sec (I am in HDMI).

I’m also having flickering as well. Even with xwayland off, just switching between blank workspaces in sway causes some flickers ocassionally - but running certain programs seems to cause more even more flickering. GTX 1060

It does seem to “work” though, if by work you mean everything is broken. Here’s some of problems and solutions I encountered:

  • Sway initially “worked” out of the box, although maybe without hardware rendering or a mouse cursor. I thought now would be the time to start tweaking my config and try moving over. While I was tweaking sway stopped working with just a black screen until I followed the instructions above by @douglas, specifically the nvidia-acceleration-overlay seemed to be key.
  • I’m using a few packages by @colemickens nixpkgs-wayland like sway, wlroots, not sure if it’s helping or not.
  • Using nixos-unstable
  • Couldn’t get gdm to work, tried toggling between nvidia, wayland, x11 but none of them work now. Just starting sway from the tty from now on.

Individual problems with applications I noticed:

  • alacritty doesn’t work: Error creating GL context; eglCreateWindowSurface failed
  • swaylock doesn’t work, I’m typing the correct password and it’s not unlocking.
  • xst has graphical artifacting, swapped it with foot.
  • ungoogled-chromium (flatpak) doesn’t work, I added --enable-features=UseOzonePlatform and --ozone-platform=wayland to ~/.var/app/com.github.Eloston.UngoogledChromium/config/chromium-flags.conf
  • codium doesn’t work, added --disable-gpu to start it but it has graphical artifacting similar to xst. The chromium flags for native wayland don’t work here for some reason.
  • Discord (flatpak) similar boat as codium.
  • Steam (flatpak) works, but sometimes it flickers and artifacts a lot. Games run accelerated, some games flicker more than others.
  • Rocket League works fine, World of Final Fantasy works but has a lot of flickering.
  • Swapped dmenu with dmenu-wayland from nixpkgs
  • ungoogled-chromium appears to be rendering webpages wrong, and is unable to play videos on some sites.
  • Firefox (flatpak) has less testing but generally seems to be working better than chromium.
  • mpv doesn’t work after switching to wayland, the gpu video output - which is the preferred output, is just a black screen. I got it to kinda work with sdl video output but it has a stange quirk where the video/audio pauses a few seconds after switching workspaces.
  • Tried ydotool instead of xdotool, just straight up doesn’t work. wlrctl seems promising.
  • Mouse feels really weird compared to x11, like maybe acceleration is on?
  • Lots of other stuff I forgot.

I’m in a weird spot right now. I’m not sure if I want to stick with the wayland that I’ve invested this time into and hope the flickering and other problems sort themselves out over time, or if I’ll switch back to x11 for who knows how long.

Edit: Formatting and hopefully making this more readable.

1 Like
43

The stuff from @douglas’s post is all in nixos-unstable. I did or merged the PRs for the related work and have been monitoring AND retested with out any nixpkgs commits to ensure no extra workarounds were needed that I didn’t mentioned.

Sway initially “worked” out of the box, although maybe without hardware rendering or a mouse cursor. I

Sounds like you didn’t use WLR_NO_HARDWARE_CURSORS and are missing one the requisite commits to enable the mesa/gbm/sway path to work properly. (Posting a sway debug log would help make this evident quickly).

And I do use ydotool/wtype and others somewhat regularly, again without a log it’s hard to say what’s wrong.

The steam/chrome/alacritty stuff are things I’ve run across too. But… I’ve sunk a STUPID number of hours into this, looking at nvidia/wlroots/etc, and I’m out of ideas for these last few things. I just can’t keep coming back to nvidia every 2 weeks and wasting hours checking the things I know to check. :frowning:

EDIT: to be clear, the feedback is great. I’ll probably make a all-up NVidia+ Wayland issue on GitHub so that other motivated folks can investigate / etc. Thanks for typing it all up @JoshuaFern

1 Like
44

You can run Alacritty using Xwayland until this problem be fixed:

WINIT_UNIX_BACKEND=x11 alacritty
1 Like
45

The stuff from @douglas’s post is all in nixos-unstable .

I thought so, maybe something funky was going on with my config.

Sounds like you didn’t use WLR_NO_HARDWARE_CURSORS

Yep, I have that now.

And I do use ydotool / wtype and others somewhat regularly, again without a log it’s hard to say what’s wrong.

Sure, the output of the terminal is unhelpful but journalctl shares some insight:

Dec 02 04:18:10 nixos-workstation kernel: input: ydotoold virtual device as /devices/virtual/input/input28
Dec 02 04:18:10 nixos-workstation systemd-logind[1017]: Failed to determine whether input device /dev/input/event21 is relevant to us: Inappropriate ioctl for device
Dec 02 04:18:10 nixos-workstation systemd[1]: Invalid unit name "actkbd@/dev/input/event21.service" escaped as "actkbd@-dev-input-event21.service" (maybe you should use systemd-escape?).
Dec 02 04:18:10 nixos-workstation systemd[1]: Starting actkbd on /dev/input/event21...
Dec 02 04:18:10 nixos-workstation systemd[1]: actkbd@-dev-input-event21.service: Deactivated successfully.
Dec 02 04:18:10 nixos-workstation systemd[1]: Started actkbd on /dev/input/event21.
Dec 02 04:18:26 nixos-workstation systemd[1]: Started Process Core Dump (PID 232160/UID 0).
Dec 02 04:18:26 nixos-workstation systemd-coredump[232161]: [🡕] Process 232149 (ydotoold) of user 0 dumped core

If I’m interpreting this right, ydotool works by sending uinput commands via systemd services. The service name ydotool uses is improperly escaped and the service never starts. Sometimes this causes a core dump of ydotoold if running. I’m just guessing though.

The steam/chrome/alacritty stuff are things I’ve run across too. But… I’ve sunk a STUPID number of hours into this, looking at nvidia/wlroots/etc, and I’m out of ideas for these last few things. I just can’t keep coming back to nvidia every 2 weeks and wasting hours checking the things I know to check. :frowning:

Yep, I’ve also had trouble with nvidia graphical artifacting on X11/i3. If I’m running MPV in an inactive workspace I can see the video peaking through the dark spots of other applications. It started happening around the time 470 came out with initial wayland support but I’ve just been dealing with it.

Thanks for all the work you’ve done on wayland. I appreciate your time greatly, and I’m sure the sentiment is shared among the community.

Maybe, just maybe, we’ve reached a precipice we can snowball down, and arewewaylandyet.com can finally give a definitive yes.

46

I got it working with vanilla unstable after I discovered the flaw I had made in my config.

I found that when switching to wayland many environment variables needed to be set, so I started blindly copying them,

      wayland.windowManager.sway.extraSessionCommands = ''
        export __GLX_VENDOR_LIBRARY_NAME=nvidia
        export _JAVA_AWT_WM_NONREPARENTING=1
        export BEMENU_BACKEND=wayland
        export CLUTTER_BACKEND=wayland
        export ECORE_EVAS_ENGINE=wayland_egl
        export ELM_ACCEL=gl
        export ELM_DISPLAY=wl
        export ELM_ENGINE=wayland_egl
        export GBM_BACKEND=nvidia-drm
        export GBM_BACKENDS_PATH=/etc/gbm
        export GDK_BACKEND=wayland
        export MOZ_DBUS_REMOTE=1
        export MOZ_ENABLE_WAYLAND=1
        export NO_AT_BRIDGE=1
        export QT_QPA_PLATFORM=wayland-egl # needs qt5.qtwayland in systemPackages
        export QT_QPA_PLATFORMTHEME="qt5ct"
        export QT_WAYLAND_DISABLE_WINDOWDECORATION="1"
        export SAL_USE_VCLPLUGIN=gtk3
        export SDL_VIDEODRIVER=wayland
        export WLR_NO_HARDWARE_CURSORS=1
        export XDG_CURRENT_DESKTOP="sway"
        export XDG_SESSION_TYPE="wayland"
      '';

It was export GBM_BACKENDS_PATH=/etc/gbm that caused the problem that @douglas’s fixes helped resolve - but actually after removing that line it does in fact work fine with just normal nixos-unstable. Oops!

Please let me know if there’s anything else here I don’t need. I know what some of it does but not all of it.