Nvidia, the bane of my existence

[bentenjamin]

I have been trying to get a working config on my horrible nvidia laptop for the past 5 days. After spending 20+ hours and testing hundreds of configs, I just cant get it working. I wouldve given up except that I got it working a single time, then updated my flake to work with home-manager, then lost the working config along the way and couldnt revert

The setup I am trying to achieve is plasma 6 + nvidia 560. The driver builds correctly but both sddm x11/wayland and startplasma-wayland always ends up with one of the following:

• black screen
• blinking underscore
• flash from [ OK ] messages to black several times, then crash to tty
• sddm fails, I can launch plasma manually, but the cursor leaves a trail around the screen and the whole desktop is unusably laggy
• boot straight to tty
  I know that the driver is doing something as I my other monitors turn on when it is loaded during boot. nvidia-smi also always returns an output with the correct driver number.

Most of my testing has been with the LTS kernel but I frequently switched to latest too. I tried other driver versions with none working, as well as every 24.05 release from august

I tried launching hyprland a couple times with no success.

I am exhausted and have seen so many error messages that I don’t even know what to include. If you don’t immediately see an issue with my config, please give me a starting point and could we try debug from there.

Here is the config file with the butt load of different options I have been trying:

{ config, pkgs, upkgs, lib, ... }:
{
  imports = [ ./boilerplate.nix ];

  #system.nixos.label = "stable-KDE_unstable-kernel-lts-nvidia-latest";

  services.xserver.videoDrivers = [ "nvidia" ];

  #hardware.graphics.enable = true;
  hardware.opengl.enable = true;
  hardware.opengl.driSupport = true;
  hardware.opengl.driSupport32Bit = true;
  
  hardware.nvidia.modesetting.enable = true;

  hardware.nvidia.powerManagement.enable = false;
  hardware.nvidia.powerManagement.finegrained = false;
  #hardware.nvidia.forceFullCompositionPipeline = true;
  hardware.nvidia.open = false;
  hardware.nvidia.nvidiaSettings = false;

  #boot.kernelPackages = upkgs.linuxPackages_latest;
  #hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.latest;
  hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.mkDriver {
    version = "560.35.03";
    sha256_64bit = "sha256-8pMskvrdQ8WyNBvkU/xPc/CtcYXCa7ekP73oGuKfH+M=";
    sha256_aarch64 = "sha256-s8ZAVKvRNXpjxRYqM3E5oss5FdqW+tv1qQC2pDjfG+s=";
    openSha256 = "sha256-/32Zf0dKrofTmPZ3Ratw4vDM7B+OgpC4p7s+RHUjCrg=";
    settingsSha256 = "sha256-kQsvDgnxis9ANFmwIwB7HX5MkIAcpEEAHc8IBOLdXvk=";
    persistencedSha256 = "sha256-E2J2wYYyRu7Kc3MMZz/8ZIemcZg68rkzvqEwFAL3fFs=";
  };

  #boot.initrd.kernelModules = [ "nvidia"
  #  "nvidia_drm" "nvidia_uvm" "nvidia_modeset" "i2c-nvidia_gpu"
  #];
  #boot.extraModulePackages = [
  #  config.boot.kernelPackages.nvidia_x11
  #];

  #boot.kernelModules = [ "hp-wmi" ];
  hardware.enableRedistributableFirmware = true;
  #hardware.enableAllFirmware = true;
  #services.thermald.enable = true;

  environment.systemPackages = with pkgs; [
    egl-wayland
    nvidia-vaapi-driver
    #vulkan-headers
  ];

  boot.kernelParams = [
    "nvidia-drm.fbdev=1"
    "module_blacklist=i915"
  ];

  boot = {
    extraModprobeConfig =
      "options nvidia "
      + lib.concatStringsSep " " [
        # nvidia assume that by default your CPU does not support PAT,
        # but this is effectively never the case in 2023
        "NVreg_UsePageAttributeTable=1"
        # This may be a noop, but it's somewhat uncertain
        "NVreg_EnablePCIeGen3=1"
        # This is sometimes needed for ddc/ci support, see
        # https://www.ddcutil.com/nvidia/
        #
        # Current monitor does not support it, but this is useful for
        # the future
        "NVreg_RegistryDwords=RMUseSwI2c=0x01;RMI2cSpeed=100"
        # When (if!) I get another nvidia GPU, check for resizeable bar
        # settings
      ];
  };

  environment.variables = {
    # these three are apparently required
    WLR_NO_HARDWARE_CURSORS = "1";
    GBM_BACKEND = "nvidia-drm";
    __GLX_VENDOR_LIBRARY_NAME = "nvidia";

    #LIBVA_DRIVER_NAME = "nvidia";
    #XDG_SESSION_TYPE = "wayland";
    #NVD_BACKEND = "direct";

    # Electron apps fix
    #NIXOS_OZONE_WL = "1";
    # Firefox VA
    #MOZ_DISABLE_RDD_SANDBOX = "1";

    # QT X11 fallback if wayland has issues, idk if both are necessary
    #QT_QPA_PLATFORM = "wayland";
    #QT_QPA_PLATFORMTHEME = "qt6ct";
  };
}

[marius]

What is the secret package needed to get the drivers to build so I can give it a go? I’m getting a lot of these:

fatal error: vulkan/vulkan.h: No such file or directory

[bentenjamin]

Check out this thread:

Unable to build nix due to nvidia drivers due or kernel 6.10(?)

Do you disable nvidiaSettings? This commit is required to get that driver version to build on stable: nvidia-settings: add vulkan-headers dependency · NixOS/nixpkgs@0384602 · GitHub

but the TLDR of it is until stable is updated, you need to disable nvidia settings:

@bentenjamin

hardware.nvidia.nvidiaSettings = false;

[eljamm]

@bentenjamin

hardware.nvidia.open = false;

This should either be set to true or removed as open is the default (and only) option from 560.28.03 onwards.

Nevermind this. It’s true that Nvidia is defaulting to open modules with 560, but it seems that the proprietary ones can still be used

[marius]

Removing the nvidiaSettings made it compile, thanks.

Wish I had more to report. For me it seems to work flawlessly so far. I updated my flatpaks so they also got 560 and loaded up Genshin Impact. Works great. Then I fired up a Steam game, also works great. (Both from X11 Cinnamon, btw).
Probably won’t do you much good, but I’ll paste my Nvidia stuff:

{ config, pkgs, fetchpatch, ... }:

{
  # Without this one an unknown monitor appears in display settings
  boot.kernelParams = [ "nvidia_drm.fbdev=1" ];

  # X11 driver
  services.xserver.videoDrivers = [ "nvidia" ];

  hardware = {

    # Enable OpenGL
    opengl.enable = true;
    opengl.driSupport = true;
    opengl.driSupport32Bit = true;

    # Modesetting is required. (ok...?)
    nvidia.modesetting.enable = true;

    # Nvidia power management. Experimental, and can cause sleep/suspend to fail.
    # Enable this if you have graphical corruption issues or application crashes after waking
    # up from sleep. This fixes it by saving the entire VRAM memory to /tmp/ instead 
    # of just the bare essentials.
    nvidia.powerManagement.enable = false;

    # Fine-grained power management. Turns off GPU when not in use.
    # Experimental and only works on modern Nvidia GPUs (Turing or newer).
    nvidia.powerManagement.finegrained = false;

    # Use the NVidia open source kernel module (not to be confused with the
    # independent third-party "nouveau" open source driver).
    # Support is limited to the Turing and later architectures. Full list of 
    # supported GPUs is at: 
    # https://github.com/NVIDIA/open-gpu-kernel-modules#compatible-gpus 
    # Only available from driver 515.43.04+
    # Currently alpha-quality/buggy, so false is currently the recommended setting.
    nvidia.open = false;

    # Enable the Nvidia settings menu,
	  # accessible via `nvidia-settings`.
#    nvidia.nvidiaSettings = true;
    nvidia.nvidiaSettings = false;
    
    # Low fps mouse in Cinnamon without this
    nvidia.forceFullCompositionPipeline = true;

    # Optionally, you may need to select the appropriate driver version for your specific GPU.
#    nvidia.package = config.boot.kernelPackages.nvidiaPackages.production;
    #nvidia.package = config.boot.kernelPackages.nvidiaPackages.stable;
    #nvidia.package = config.boot.kernelPackages.nvidiaPackages.beta;
    nvidia.package = config.boot.kernelPackages.nvidiaPackages.mkDriver {
      version = "560.35.03";
      sha256_64bit = "sha256-8pMskvrdQ8WyNBvkU/xPc/CtcYXCa7ekP73oGuKfH+M=";
      sha256_aarch64 = "sha256-s8ZAVKvRNXpjxRYqM3E5oss5FdqW+tv1qQC2pDjfG+s=";
      openSha256 = "sha256-/32Zf0dKrofTmPZ3Ratw4vDM7B+OgpC4p7s+RHUjCrg=";
      settingsSha256 = "sha256-kQsvDgnxis9ANFmwIwB7HX5MkIAcpEEAHc8IBOLdXvk=";
      persistencedSha256 = "sha256-E2J2wYYyRu7Kc3MMZz/8ZIemcZg68rkzvqEwFAL3fFs=";
    };

  };
}

Next I’ll be testing out the open kernel modules.
And also, my system is a desktop machine with a 4090 and no iGPU. Maybe you are experiencing blacklisting issues or something?

[TLATER]

There are other people struggling with sddm here: Sddm fails to launch

Can you share your logs?

You’re also notably missing the modesetting option which I think is required for wayland.

My suspicion is indeed that you’re not disabling the iGPU properly, laptops almost certainly have more than one GPU. You’re both forcibly configuring nvidia with forced kernel modules and not disabling intel/amd, so this will probably cause issues.

If you want to disable your non-nvidia GPU, take a look at this nixos-hardware module: nixos-hardware/common/gpu/intel/disable.nix at 95c3dfe6ef2e96ddc1ccdd7194e3cda02ca9a8ef · NixOS/nixos-hardware · GitHub

Check out the others while you’re at it, there may be one for your laptop, and maybe you actually do want to use both your GPUs and not burn through battery like it’s nothing. The nvidia modules are helpful anyway, since they set up a good basic config that should work™.

Also, while I’m at it, you almost certainly want to enable the powermanagement service, without it sleep won’t work. Finegrained is useful on laptops, too, without it any dGPU/iGPU work splitting is pointless. The env variables are wlroots-specific, so pointless for plasma. In general a lot of your config is redundant at best, you should definitely simplify it a lot to get it working before tweaking details like this.

[bentenjamin]

deep breath

…sigh…

I have just experienced tiers of psychic damage I didn’t know existed. The driver was a red herring the entire time.

Thank you @TLATER for the great help. You are right that modesetting is required for wayland, which I had enabled

@bentenjamin

hardware.nvidia.modesetting.enable = true;

It was silly of me to not include the quirks of my laptop. It is an HP OMEN Laptop 15-ce00ni which has the intel iGPU disabled by HP and is unusable. nix run github:eclairevoyant/pcids returns:

PCI:1:0:0
	NVIDIA Corporation [10de]
	GP106BM [GeForce GTX 1060 Mobile 6GB] [1c60]

but out of desperation, and having no idea whether nix was falsely trying to load intel drivers, I did try to disable the iGPU as you have said:

@bentenjamin

"module_blacklist=i915"

Something interesting I see is that the wiki and the nixos-hardware repo are desynced. The wiki states to just add boot.kernelParams = [ "module_blacklist=i915" ]; but the repo has

  boot.blacklistedKernelModules = lib.mkDefault [ "i915" ];
  # KMS will load the module, regardless of blacklisting
  boot.kernelParams = lib.mkDefault [ "i915.modeset=0" ];

I don’t know which one is required, but I thought I’d point it out.

My specific device is not in the nixos hardware repo, but I had tried to use some options that I saw other HP laptops using

@bentenjamin

  #boot.kernelModules = [ "hp-wmi" ];

Though as you will see soon this wasn’t the issue. Now that I have a desktop I’ll start testing quirks and see what else there is for this machine.

So…on to the story

I installed nixos onto a separate drive as I want to set up an environment without touching my current system. I was unable to get into desktop because there’s no drivers. I booted into my main system, did research into getting nix set up, created config files and copied them over. Queue this past week of pain. Turns out “unbreakable system” doesn’t include filling up your initrd while testing and nuking the only working generation with nix-garbage-collect -d, hah couldn’t be me, anyway.

After reading your reply I tried the extra options from the hardware repo with no success. Reading the post you linked about their being issues with SDDM in general, I figured I’d try an as ‘default’ setup as I could. I switched kde and sddm for gnome and lightDM. Unsurprisingly, it doesn’t work, so I go to get info from the logs to post here.

Now,I have been on linux for a few years now with different distributions. I still have lots to learn but I have been into journalctl a few times, and seen lots of warnings and errors that log on every boot but never affect anything. Unfortunately this means I have become jaded to warning/error messages of things that I don’t think are the issue.

I checked the lightDM logs, slap them into startpage, land on

Stuck at `[ ok ] started x11 server`

“Detected unsafe path” messages was in orange. I think it means they were warning ● systemd-tmpfiles-setup.service - Create Volatile Files and Directories Loaded: loaded (/etc/systemd/system/systemd-tmpfiles-setup.service; enabled; vendor preset: enabled) Active: active (exited) since Fri 2022-09-16 21:17:02 UTC; 3min 18s ago Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) Process: 475 ExecStart=systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev (c…

And the solution–

Stuck at `[ ok ] started x11 server`

Your root directory is owned by uid 1000. Probably an artefact of how you mounted your root drive. Mount it, and sudo chown -R root:root /mnt. I guess systemd chickens out and doesnt’t create these directories because of the root dir permissions.

…wait…

“I booted into my main system … and copied them over.” - bentenjamin

MOTHER FU- (ノಠ益ಠ)ノ彡┻━┻

…sigh…
So after all of this agony the issue was that various permissions were messed up when copying/saving configs from my other DE. And you @TLATER the man(?), the myth, the legend, had already solved my issue, 2 years ago.

Reinstalling nixos and using the most basic config is running perfectly:

{ config, pkgs, ... }:
{
 services.xserver.videoDrivers = [ "nvidia" ];

  #hardware.graphics.enable = true;
  hardware.opengl = {
    enable = true;
    driSupport = true;
    driSupport32Bit = true;
  };
  
  hardware.nvidia = {
    modesetting.enable = true;
    powerManagement.enable = true;
    open = false;
    nvidiaSettings = false;

    package = config.boot.kernelPackages.nvidiaPackages.mkDriver {
      version = "560.35.03";
      sha256_64bit = "sha256-8pMskvrdQ8WyNBvkU/xPc/CtcYXCa7ekP73oGuKfH+M=";
      sha256_aarch64 = "sha256-s8ZAVKvRNXpjxRYqM3E5oss5FdqW+tv1qQC2pDjfG+s=";
      openSha256 = "sha256-/32Zf0dKrofTmPZ3Ratw4vDM7B+OgpC4p7s+RHUjCrg=";
      settingsSha256 = "sha256-kQsvDgnxis9ANFmwIwB7HX5MkIAcpEEAHc8IBOLdXvk=";
      persistencedSha256 = "sha256-E2J2wYYyRu7Kc3MMZz/8ZIemcZg68rkzvqEwFAL3fFs=";
    };
  };
}

It is frustrating that I couldn’t look past the nvidia/DM error logs to find the solution, but thank you so much for everything you do for the community <3

[TLATER]

@bentenjamin

Turns out “unbreakable system” doesn’t include filling up your initrd while testing and nuking the only working generation with nix-garbage-collect -d, hah couldn’t be me, anyway.

For future reference, that could have been fixed pretty easily by booting up the installer, mounting the drives, using nixos-enter to chroot into your system, and rerunnig nixos-rebuild. You only deleted the old boot entries and kernels associated with the activation script for your system, the kernel would still have been in the nix store, so you could even have copied that over and manually written a boot entry.

Using -d on a garbage collect without first rebooting is definitely playing with fire, though that still only causes issues if you use nixos-rebuild switch instead of boot too.

Definitely a paper cut, regardless, wonder if this could be improved. Users should maybe not be able to turn off all the safety switches and take off their seat belts?

---

I feel your pain though, that’s an increidbly niche edge case. Glad I’ve stumbled across it before and my backlog of replies to seemingly impossible questions is actually helpful to folks x)