Nvidia Fails to build

[Tetrodotoxin]

New to nix but I’m moderately experienced linux,
I may be missing something small or silly here, tried updating my kernel to no avail
Trying to install the nvidia drivers based on the wiki page,but a niixos-rebuild fails at the end gives me:

error: builder for '/nix/store/hpw9lmk7d2w3aksjvjk2li0zhzrv7jpv-nvidia-x11-535.86.05-6.5.7.drv' failed with exit code 2
error: 1 dependencies of derivation '/nix/store/3d81yf38hz9zrb9lz9kpq4ay6dpdq6ac-etc.drv' failed to build
error: 1 dependencies of derivation '/nix/store/j6h2bfbnhw96vxv260vvjb23b6fjvbrg-firmware.drv' failed to build
error: 1 dependencies of derivation '/nix/store/asi7wwz74bw3sr8xw4vvlmri440jf3b9-system-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/wqfk14jwydmpb8v21q5vrpylh7a3b7py-nixos-system-nixos-23.05.4407.80c1aab72515.drv' failed to build

Here’s the link to the full output: [big_boss@nixos:~]$ sudo nixos-rebuild switch | tee log.txt building Nix...b - Pastebin.com
Here’s my Nixinfo:

 - system: `"x86_64-linux"`
 - host os: `Linux 6.5.7, NixOS, 23.05 (Stoat), 23.05.4407.80c1aab72515`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.5`
 - channels(big_boss): `""`
 - channels(root): `"nixos-23.05"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

and my nix.configuration

# Edit this configuration file to define what should be installed on
# your system.  Help is available in the configuration.nix(5) man page
# and in the NixOS manual (accessible by running ‘nixos-help’).

{ config, pkgs, ... }:

{


  # Enable OpenGL
  hardware.opengl = {
    enable = true;
    driSupport = true;
    driSupport32Bit = true;
  };

  # Load nvidia driver for Xorg and Wayland
  services.xserver.videoDrivers = ["nvidia"];

  hardware.nvidia = {

    # Modesetting is required.
    modesetting.enable = true;

    # Nvidia power management. Experimental, and can cause sleep/suspend to fail.
    powerManagement.enable = false;
    # Fine-grained power management. Turns off GPU when not in use.
    # Experimental and only works on modern Nvidia GPUs (Turing or newer).
    powerManagement.finegrained = false;

    # Use the NVidia open source kernel module (not to be confused with the
    # independent third-party "nouveau" open source driver).
    # Support is limited to the Turing and later architectures. Full list of 
    # supported GPUs is at: 
    # https://github.com/NVIDIA/open-gpu-kernel-modules#compatible-gpus 
    # Only available from driver 515.43.04+
    # Do not disable this unless your GPU is unsupported or if you have a good reason to.
    open = true;

    # Enable the Nvidia settings menu,
        # accessible via `nvidia-settings`.
    nvidiaSettings = true;

    # Optionally, you may need to select the appropriate driver version for your specific GPU.
    package = config.boot.kernelPackages.nvidiaPackages.stable;
  };


  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
    ];

  # Bootloader.
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  networking.hostName = "nixos"; # Define your hostname.
  # networking.wireless.enable = true;  # Enables wireless support via wpa_supplicant.
  boot.kernelPackages = pkgs.linuxPackages_latest;
  # Configure network proxy if necessary
  # networking.proxy.default = "http://user:password@proxy:port/";
  # networking.proxy.noProxy = "127.0.0.1,localhost,internal.domain";

  # Enable networking
  networking.networkmanager.enable = true;

  # Set your time zone.
  time.timeZone = "America/New_York";

  # Select internationalisation properties.
  i18n.defaultLocale = "en_US.UTF-8";

  i18n.extraLocaleSettings = {
    LC_ADDRESS = "en_US.UTF-8";
    LC_IDENTIFICATION = "en_US.UTF-8";
    LC_MEASUREMENT = "en_US.UTF-8";
    LC_MONETARY = "en_US.UTF-8";
    LC_NAME = "en_US.UTF-8";
    LC_NUMERIC = "en_US.UTF-8";
    LC_PAPER = "en_US.UTF-8";
    LC_TELEPHONE = "en_US.UTF-8";
    LC_TIME = "en_US.UTF-8";
  };

  # Enable the X11 windowing system.
  services.xserver.enable = true;

  # Enable the GNOME Desktop Environment.
  services.xserver.displayManager.gdm.enable = true;
  services.xserver.desktopManager.gnome.enable = true;

  # Configure keymap in X11
  services.xserver = {
    layout = "us";
    xkbVariant = "";
  };

  # Enable CUPS to print documents.
  services.printing.enable = true;

  # Enable sound with pipewire.
  sound.enable = true;
  hardware.pulseaudio.enable = false;
  security.rtkit.enable = true;
  services.pipewire = {
    enable = true;
    alsa.enable = true;
    alsa.support32Bit = true;
    pulse.enable = true;
    # If you want to use JACK applications, uncomment this
    #jack.enable = true;

    # use the example session manager (no others are packaged yet so this is enabled by default,
    # no need to redefine it in your config for now)
    #media-session.enable = true;
  };

  # Enable touchpad support (enabled default in most desktopManager).
  # services.xserver.libinput.enable = true;

  # Define a user account. Don't forget to set a password with ‘passwd’.
  users.users.big_boss = {
    isNormalUser = true;
    description = "Big Boss";
    extraGroups = [ "networkmanager" "wheel" ];
    packages = with pkgs; [
      firefox
    #  thunderbird
    ];
  };

  # Allow unfree packages
  nixpkgs.config.allowUnfree = true;

  # List packages installed in system profile. To search, run:
  # $ nix search wget
  environment.systemPackages = with pkgs; [
    pkgs.vim # Do not forget to add an editor to edit configuration.nix! The Nano editor is also installed by default.
        pkgs.tmux  
#  wget
  ];

  # Some programs need SUID wrappers, can be configured further or are
  # started in user sessions.
  # programs.mtr.enable = true;
  # programs.gnupg.agent = {
  #   enable = true;
  #   enableSSHSupport = true;
  # };

  # List services that you want to enable:

  # Enable the OpenSSH daemon.
  # services.openssh.enable = true;
 Edit this configuration file to define what should be installed on
# your system.  Help is available in the configuration.nix(5) man page
# and in the NixOS manual (accessible by running ‘nixos-help’).

{ config, pkgs, ... }:

{


  # Enable OpenGL
  hardware.opengl = {
    enable = true;
    driSupport = true;
    driSupport32Bit = true;
  };

  # Load nvidia driver for Xorg and Wayland
  services.xserver.videoDrivers = ["nvidia"];

  hardware.nvidia = {

    # Modesetting is required.
    modesetting.enable = true;

    # Nvidia power management. Experimental, and can cause sleep/suspend to fail.
    powerManagement.enable = false;
    # Fine-grained power management. Turns off GPU when not in use.
    # Experimental and only works on modern Nvidia GPUs (Turing or newer).
    powerManagement.finegrained = false;

    # Use the NVidia open source kernel module (not to be confused with the
    # independent third-party "nouveau" open source driver).
    # Support is limited to the Turing and later architectures. Full list of 
    # supported GPUs is at: 
    # https://github.com/NVIDIA/open-gpu-kernel-modules#compatible-gpus 
    # Only available from driver 515.43.04+
    # Do not disable this unless your GPU is unsupported or if you have a good reason to.
    open = true;

    # Enable the Nvidia settings menu,
        # accessible via `nvidia-settings`.
    nvidiaSettings = true;

    # Optionally, you may need to select the appropriate driver version for your specific GPU.
    package = config.boot.kernelPackages.nvidiaPackages.stable;
  };


  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
    ];

  # Bootloader.
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  networking.hostName = "nixos"; # Define your hostname.
  # networking.wireless.enable = true;  # Enables wireless support via wpa_supplicant.
  boot.kernelPackages = pkgs.linuxPackages_latest;
  # Configure network proxy if necessary
  # networking.proxy.default = "http://user:password@proxy:port/";
  # networking.proxy.noProxy = "127.0.0.1,localhost,internal.domain";

  # Enable networking
  networking.networkmanager.enable = true;

  # Set your time zone.
  time.timeZone = "America/New_York";

  # Select internationalisation properties.
  i18n.defaultLocale = "en_US.UTF-8";

  i18n.extraLocaleSettings = {
    LC_ADDRESS = "en_US.UTF-8";
    LC_IDENTIFICATION = "en_US.UTF-8";
    LC_MEASUREMENT = "en_US.UTF-8";
    LC_MONETARY = "en_US.UTF-8";
    LC_NAME = "en_US.UTF-8";
    LC_NUMERIC = "en_US.UTF-8";
    LC_PAPER = "en_US.UTF-8";
    LC_TELEPHONE = "en_US.UTF-8";
    LC_TIME = "en_US.UTF-8";
  };

  # Enable the X11 windowing system.
  services.xserver.enable = true;

  # Enable the GNOME Desktop Environment.
  services.xserver.displayManager.gdm.enable = true;
  services.xserver.desktopManager.gnome.enable = true;

  # Configure keymap in X11
  services.xserver = {
    layout = "us";
    xkbVariant = "";
  };

  # Enable CUPS to print documents.
  services.printing.enable = true;

  # Enable sound with pipewire.
  sound.enable = true;
  hardware.pulseaudio.enable = false;
  security.rtkit.enable = true;
  services.pipewire = {
    enable = true;
    alsa.enable = true;
    alsa.support32Bit = true;
    pulse.enable = true;
    # If you want to use JACK applications, uncomment this
    #jack.enable = true;

    # use the example session manager (no others are packaged yet so this is enabled by default,
    # no need to redefine it in your config for now)
    #media-session.enable = true;
  };

  # Enable touchpad support (enabled default in most desktopManager).
  # services.xserver.libinput.enable = true;

  # Define a user account. Don't forget to set a password with ‘passwd’.
  users.users.big_boss = {
    isNormalUser = true;
    description = "Big Boss";
    extraGroups = [ "networkmanager" "wheel" ];
    packages = with pkgs; [
      firefox
    #  thunderbird
    ];
  };

  # Allow unfree packages
  nixpkgs.config.allowUnfree = true;

  # List packages installed in system profile. To search, run:
  # $ nix search wget
  environment.systemPackages = with pkgs; [
    pkgs.vim # Do not forget to add an editor to edit configuration.nix! The Nano editor is also installed by default.
        pkgs.tmux  
#  wget
  ];

  # Some programs need SUID wrappers, can be configured further or are
  # started in user sessions.
  # programs.mtr.enable = true;
  # programs.gnupg.agent = {
  #   enable = true;
  #   enableSSHSupport = true;
  # };

  # List services that you want to enable:

  # Enable the OpenSSH daemon.
  # services.openssh.enable = true;

[Tetrodotoxin]

After rebooting a few times it just started working?
Not really sure what happened here.

[TLATER]

@Tetrodotoxin

boot.kernelPackages = pkgs.linuxPackages_latest;

The nvidia drivers are a third party kernel module and therefore aren’t always compatible with the kernel in linuxPackages_latest. Usually if you see build failures for the nvidia package that’s the cause. I’d wager you updated your channels at some point and a version bump happened in the mean time.

NixOS stable tends to package a fairly old nvidia driver, so patches that make it compatible with the most recent kernels lag behind quite a bit. Since any nvidia driver version bump is technically a breaking change (they have no minor or patch releases), maintainers are usually hesitant to merge version bumps (though in the past I’ve managed to twist their arms after a week or two of discussion).

Personally nowadays to avoid this I use unstable for my kernel + driver, that minimizes the time the two end up out of sync, and brings me to the most recent nvidia driver that I can reasonably run. Linux’ don’t-break-userspace policy means that this is relatively sane, too, and because it’s so low-level I think I don’t even have a second copy of glibc.

I know of an upcoming change in the kernel to prevent nvidia from using a gpl condom that will likely break even my setup, though.

In the long term, I’ll probably switch to the LTS kernel version (linuxPackages_default, albeit still with the xanmod patches), but some of my hardware still isn’t supported by Linux < 6.0. And in the very long term I’ll probably switch to a different GPU vendor entirely, all the nvidia headaches with kernel compatibility and wayland problems aren’t worth it.

There were some discussions around how third party modules could reasonably be made to work with non-LTS kernels most of the time, but there’s not enough drive behind NixOS to support kernels not supported by Linux upstream, and therefore versions needed for third party modules can’t be packaged for security reasons. This means that whenever there’s a kernel update that breaks your module you’re SOOL, and have to hope the LTS release works on your hardware (or that the third party modules’ vendor has released an update since and either use unstable to apply it or get it backported to stable).

tl;dr: Don’t use nvidia with linuxPackages_latest, especially when you’re using NixOS stable.

[Tetrodotoxin]

Thanks for the well thought out reply, I’ll keep that in mind going forward.

[wamserma]

I usually have the latest zfs-compatible Kernel and this work pretty well with the NVIDIA driver (I’m running 23.05).

From my config:

let
   # use latest kernel, but keep ZFS compatibility, actual version is at most:
   # nix eval nixos#legacyPackages.x86_64-linux.zfs.latestCompatibleLinuxPackages.kernel.version
     myKernelVersion = pkgs:
     let
       zfs_version = pkgs.zfs.latestCompatibleLinuxPackages.kernel.version;
       latest_kernel_version = pkgs.linuxPackages_latest.kernel.version;
       preferred_version =
         if pkgs.lib.versionAtLeast zfs_version latest_kernel_version
         then pkgs.linuxPackages_latest
         else pkgs.zfs.latestCompatibleLinuxPackages;
       enforce_version = {version ? preferred_version}: version;
     in
       # or whatever NVIDIA compiles with
       enforce_version { }; # enforce_version { version = pkgs.linuxPackages_6_0; };
in
   boot.kernelPackages = myKernelVersion pkgs;
   hardware.nvidia.package = (myKernelVersion pkgs_unfree).nvidiaPackages.legacy_470;