Nvidia prime offload mode on dell xps 7590

Hey! I’m trying to get nvidia prime offload working on a Dell XPS 15" 7590.

It comes with

  • CPU: i7-9750H
  • GPU: GeForce GTX 1650 Mobile / Max-Q] (rev a1)
  1. First question:

When referencing the wiki page on nvidia, it seems that the card should power completely off when not in use. I previously used Arch Linux, where I used Optimus-Manager to manually turn it off. This resulted in a power draw of ~2W. Right now NixOS is drawing ~8-10W. To be honest, I do not know if this is what should be expected, but it seems too high.

When I disable the card entirely (in NixOs), I get a draw of ~2W, but I would love if the offload mode could yield the same results. I have tried the following things:

  • Doing exactly what the Nvidia wiki page proposed
  • using latest kernel
  • add “modesetting” to video drivers
  • using lightdm instead of gdm
  • adding options nvidia "NVreg_DynamicPowerManagement=0x02" to modprobe, both in nix boot.extraModprobeConfig and imperatively in /etc/modprobe.d/nvidia.conf
  • setting hardware.nvidia.powerManagement.enable to true
  • adding udev rules from nvidia’s site:
services.udev.extraRules = ''
                           ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x0c0330", ATTR{remove}="
                           ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x0c8000", ATTR{remove}="1"
                           ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x040300", ATTR{remove}="1"
                           ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="auto"
                           ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="auto"
                           ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="on"
                           ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="on"
                           '';

Has anyone with a similar setup got this working?

  1. Second question

I can not get the nvidia card working at all in offload mode, when using GDM.
nvidia-offload glxinfo | grep vendor returns

X Error of failed request:  BadValue (integer parameter out of range for operation)
  Major opcode of failed request:  152 (GLX)
  Minor opcode of failed request:  24 (X_GLXCreateNewContext)
  Value in failed request:  0x0
  Serial number of failed request:  39
  Current serial number in output stream:  40

It works when using lightdm however. Have anyone made it work with GDM?

Thanks!

(Also asked on reddit)

Okay, I found a solution! Shoutout to this link. I believe it still only works with lightdm, no gdm. This is what fixed it

services.xserver.videoDrivers = [ "nvidia" ];
  hardware.nvidia = {
    prime = {
      offload.enable = true;
      # Bus ID of the Intel GPU. You can find it using lspci, either under 3D or VGA
      intelBusId = "PCI:0:2:0";
      # Bus ID of the NVIDIA GPU. You can find it using lspci, either under 3D or VGA
      nvidiaBusId = "PCI:1:0:0";
    };
    #powerManagement.enable = true;
  };

boot.extraModprobeConfig = "options nvidia \"NVreg_DynamicPowerManagement=0x02\"\n";
  services.udev.extraRules = ''
  # Remove NVIDIA USB xHCI Host Controller devices, if present
  ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x0c0330", ATTR{remove}="1"
  
  # Remove NVIDIA USB Type-C UCSI devices, if present
  ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x0c8000", ATTR{remove}="1"
  
  # Remove NVIDIA Audio devices, if present
  ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x040300", ATTR{remove}="1"
  
  # Enable runtime PM for NVIDIA VGA/3D controller devices on driver bind
  ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="auto"
  ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="auto"
  
  # Disable runtime PM for NVIDIA VGA/3D controller devices on driver unbind
  ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="on"
  ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="on"
  '';  

And I have some other nvidia related settings around, I havent had time to bisect it yet, but just to have it out there!

boot.blacklistedKernelModules =  [ "nouveau" ];
services.tlp = {
      enable = true;
      settings = {
        RUNTIME_PM_DRIVER_BLACKLIST = "nouveau mei_me";
      };
    };
# cpu stuff
  boot.initrd.kernelModules = [ "intel_agp" "i915" ];
  hardware.enableRedistributableFirmware = true;
  hardware.cpu.intel.updateMicrocode = true;
  hardware.opengl.extraPackages = with pkgs; [
    vaapiIntel
    vaapiVdpau
    libvdpau-va-gl
    intel-media-driver
  ];

This solution is not working for me (Dell XPS 7590 on NixOS 21.11).

I also tried to leverage existing modules from nixos-hardware, with

/etc/nixos/configuration.nix

  imports =
    [ # Include the results of the hardware scan.
      <nixos-hardware/dell/xps/15-7590>
      ./nvidia.nix
      ./hardware-configuration.nix
  ]
...

/etc/nixos/nvidia.nix:

{ lib, pkgs, ... }:
{
  imports = [
    <nixos-hardware/common/gpu/nvidia.nix>
  ];

  boot.kernelParams = lib.mkDefault [ "acpi_rev_override" ];

  services.thermald.enable = lib.mkDefault true;

  hardware.nvidia.prime = {
    # Bus ID of the Intel GPU.
    intelBusId = lib.mkDefault "PCI:0:2:0";
    # Bus ID of the NVIDIA GPU.
    nvidiaBusId = lib.mkDefault "PCI:1:0:0";
  };
}

with the content of nvidia.nix mostly taken from <nixos-hardware/dell/xps/15-9500/nvidia>.

I have also updated my config a bit, you can see it here! See my configration.nix, it works for me. And see nvidia-offload.nix, which has some shell scripts you can use to check if it works. I have included them en my environment packages so I can just run nvidia-offload-status to get an overview of what works and if my GPU is suspended.