NVIDIA RTX 4070 GPU not being used in games

Hi all, I recently built a new PC and decided to try NixOS instead of my old Arch + Windows dual boot setup. However, I have been unable to get games to work well. I am able to launch them, but they run extremely poorly.

On using MangoHUD, it seems like my GPU utilization is at 0%. I’ve ensured that MangoHUD is indeed looking at my RTX 4070 by having it display gpu_name, and games that were working well with a much much older GPU are now struggling to run on my new machine. It may be relevant that I am using Hyprland with Wayland. Here are the relevant parts of my /etc/nixos/configuration.nix.

# despite having the NVIDIA modeset, my PC is stuck on booting without this 
boot.kernelParams = [ "nomodeset" ];

nixpkgs.config.allowUnfree = true;
environment.systemPackages = with pkgs; [
  nvidia-vaapi-driver

  # tried because of https://github.com/NixOS/nixpkgs/issues/108598#issuecomment-917682592
  # but either way, no difference
  # (steam.override {
  #   extraProfile = ''
  #     export VK_ICD_FILENAMES=${config.hardware.nvidia.package}/share/vulkan/icd.d/nvidia_icd.json:${config.hardware.nvidia.package.lib32}/share/vulkan/icd.d/nvidia_icd32.json:$VK_ICD_FILENAMES
  #   '';
  # })
];

# most of the NVIDIA stuff is lifted from https://nixos.wiki/wiki/Nvidia
hardware.opengl = {
  enable = true;
  driSupport = true;
  driSupport32Bit = true;
  extraPackages = with pkgs; [
    vaapiVdpau
    libvdpau-va-gl
  ];
  extraPackages32 = with pkgs.pkgsi686Linux; [ libva ];
  setLdLibraryPath = true;
};

services.xserver = {
  enable = true;
  videoDrivers = [ "nvidia" ];
  displayManager.gdm = {
    enable = true;
    wayland = true;
  };
};

hardware.nvidia = {
  modesetting.enable = true;
  open = false;
  nvidiaSettings = true;
  package = config.boot.kernelPackages.nvidiaPackages.stable;
};

programs.hyprland = {
  enable = true;
  nvidiaPatches = true;
  xwayland = {
    hidpi = true;
    enable = true;
  };
};

# either way, gamescope doesn't make a difference
# programs.gamescope.enable = true;
programs.steam = {
  enable = true;
  remotePlay.openFirewall = true;
  dedicatedServer.openFirewall = true;
};

environment.sessionVariables = {
  LIBVA_DRIVER_NAME = "nvidia";
  XDG_SESSION_TYPE = "wayland";
  GBM_BACKEND = "nvidia-drm";
  __GLX_VENDOR_LIBRARY_NAME = "nvidia";
  WLR_NO_HARDWARE_CURSORS = "1";
  NIXOS_OZONE_WL = "1";
  MOZ_ENABLE_WAYLAND = "1";

  # setting the SDL_VIDEODRIVER to wayland or x11 doesn't make a difference
  # a lot of these came from https://github.com/NixOS/nixpkgs/issues/162562#issuecomment-1523177264
  # SDL_VIDEODRIVER = "wayland";
  _JAVA_AWT_WM_NONREPARENTING = "1";
  CLUTTER_BACKEND = "wayland";
  WLR_RENDERER = "vulkan";
  XDG_CURRENT_DESKTOP = "Hyprland";
  XDG_SESSION_DESKTOP = "Hyprland";
  GTK_USE_PORTAL = "1";
  NIXOS_XDG_OPEN_USE_PORTAL = "1";
  GDK_BACKEND = "wayland,x11";
  QT_QPA_PLATFORM = "wayland;xcb";
  ENABLE_VKBASALT = "1";
};

I included my Steam configuration, but my issues don’t just come from Steam: when launching programs like glxgears or vkcube or Minecraft, my GPU utilization still sits at 0%. Interestingly, on programs that display what graphics device they are using (like vkcube), they say they are using my RTX 4070. Regardless though, their performance ends up still being much poorer than it should be were they actually utilizing the GPU to the extent they should be.

For additional info, here is the first part of the output of nvidia-smi, which seems to clearly indicate some sort of problem. The GPU-Util field is unsurprisingly always 0%.

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.86.05              Driver Version: 535.86.05    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4070        Off | 00000000:01:00.0  On |                  N/A |
|ERR!   52C    P0              20W / 200W |   1268MiB / 12282MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

My NVIDIA drivers version is 535.86.05 and my NVML Version is 12.535.86.05. I tried installing newer NVIDIA drivers from their website (the version would be 535.98), but running their .run file gave me the output

Verifying archive integrity... OK
Uncompressing NVIDIA Accelerated Graphics Driver for Linux-x86_64 535.98......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
./NVIDIA-Linux-x86_64-535.98.run: line 732: ./nvidia-installer: cannot execute: required file not found

I don’t know how to run this file in NixOS, so I’m uncertain as to whether newer drivers would fix my issue.

Some odd things that stand out to me.

  • I cannot boot without the nomodeset kernel parameter, even though I have the NVIDIA modeset enabled in my config. This suggests that maybe there is some issue with the installed NVIDIA drivers. The nvidia-smi output also seems to indicate something is wrong.
  • My sole monitor is connected through my one of my GPU’s DisplayPort ports, so the GPU isn’t doing nothing.
  • I tried testing with Steam through the nixpkgs and Steam through FlatPak, and I tried lots of different versions of Proton, but regardless, launched games would run slow; the issue lies not in Steam anways as I have problems with Minecraft, a non-Steam game. Minecraft interestingly can see the RTX 4070, and it says it is utilizing 100% or nearly 100% of the GPU, but it still ends up being quite laggy, and at the same time, MangoHUD still claims the CPU is doing some work but the GPU is at 0% load.

In summary, my GPU is not being utilized in games seemingly. I am running a new NixOS system with an NVIDIA RTX 4070 on Hyprland and Wayland. MangoHUD and nvidia-smi say that my GPU utilization is at 0%, and while games claim to use my GPU, their performance doesn’t seem to indicate it. I believe there is some drivers issue, but I don’t know what the issue is or how to fix it, and I would greatly appreciate help so I don’t need to resort to dual-booting again or changing significant aspects of my system.

Use the drivers from the nixpkgs repo, it’s quite difficult to run software not compiled with nix under NixOS. If you want a more up-to-date driver, grab one from unstable, and if that’s somehow still not recent enough override the inputs of the package.

I don’t think anything is wrong with the drivers, it’s most likely your configuration. Setting nomodeset is a huge red flag, usually setting it breaks a lot, since you’re usually fall back to unaccelerated rendering (which would be what you’re seeing…). It’s intended as a last-ditch effort to get a system to boot for debugging purposes, not for gaming. I’m surprised your system doesn’t just freeze when launching stuff.

If you need it to boot that points to a big hardware compatibility issue, which should not be a thing for 40 series cards anymore. Did you need to set it to boot the installer as well or was this a post-installation thing?

Frankly, I don’t think setting hardware.nvidia.modesetting.enable = true; when you also explicitly disable modesetting with nomodeset makes any sense. Maybe it will all just work when you remove both of those, and we can try to get modesetting to work from there?

1 Like

Thanks for the suggestions! I needed the nomodeset kernel parameter previously for the boot installer as well. I disabled my CPU integrated graphics by force through the BIOS (the option was set to auto and I set it to disabled), and now I’m able to boot without the nomodeset kernel parameter (but I have kept the hardware.nvidia.modesetting.enable = true;)! Unfortunately, despite that aspect being resolved, I was still having issues in terms of performance and GPU utilization. Games were still running poorly, and nvidia-smi and MangoHUD were still showing 0% GPU utilization. To try and remedy this, I switched to unstable which does have the latest drivers that I was attempting to install manually. However, that still has failed to fix GPU utilization, and it’s still stuck at 0%.

Did you try the setup explained on the wiki? Nvidia - NixOS Wiki

Works for me© with an external GPU, which is a similar case as a discrete GPU.

1 Like

Yes, they are :wink:

Well that sucks. Anything in dmesg, journalctl? How about your hyprland logs?

2 Likes

I appreciate all the help!

Looking through dmesg, I didn’t see anything out of place. Here are the outputs for dmesg | grep -i nvidia.

[    0.000000] Command line: initrd=\efi\nixos\rzgcs4y2b9mwb9a65g3c8xykdqq2lkby-initrd-linux-6.1.46-initrd.efi init=/nix/store/p4s0hpk3yr4gdl91rgrqvs4lj1j21808-nixos-system-photon-23.11pre516114.d680ded26da5/init loglevel=4 nvidia-drm.modeset=1
[    0.027706] Kernel command line: initrd=\efi\nixos\rzgcs4y2b9mwb9a65g3c8xykdqq2lkby-initrd-linux-6.1.46-initrd.efi init=/nix/store/p4s0hpk3yr4gdl91rgrqvs4lj1j21808-nixos-system-photon-23.11pre516114.d680ded26da5/init loglevel=4 nvidia-drm.modeset=1
[    3.852342] nvidia: loading out-of-tree module taints kernel.
[    3.852350] nvidia: module license 'NVIDIA' taints kernel.
[    3.900308] nvidia-nvlink: Nvlink Core is being initialized, major device number 244
[    3.901171] nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
[    3.948307] NVRM: loading NVIDIA UNIX x86_64 Kernel Module  535.98  Tue Aug  1 21:42:05 UTC 2023
[    3.968171] nvidia_uvm: module uses symbols nvUvmInterfaceDisableAccessCntr from proprietary module nvidia, inheriting taint.
[    3.986248] nvidia-uvm: Loaded the UVM driver, major device number 242.
[    3.992388] nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms  535.98  Tue Aug  1 21:40:14 UTC 2023
[    4.006315] [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
[    4.298024] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/0000:01:00.1/sound/card0/input23
[    4.298072] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/0000:01:00.1/sound/card0/input24
[    4.298120] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:01.1/0000:01:00.1/sound/card0/input25
[    4.298170] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:01.1/0000:01:00.1/sound/card0/input26
[    5.169724] [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 0

None of this stands out to me or seems like it points to any issues. Similarly, nothing of note stood out to me from my Hyprland logs.

The outputs of journalctl maybe seem to possibly indicate some things aren’t working. Following this paragraph is the output of journalctl | grep "Aug 22" | grep -i nvidia. I don’t really know how to interpret this all, but the last two lines seem to indicate something going wrong, and both of the (udev-worker)[829] lines return a non-zero exit code too. However, whether these are serious issues or the cause of my problems, I don’t know.

Aug 22 07:40:08 photon kernel: Command line: initrd=\efi\nixos\rzgcs4y2b9mwb9a65g3c8xykdqq2lkby-initrd-linux-6.1.46-initrd.efi init=/nix/store/p4s0hpk3yr4gdl91rgrqvs4lj1j21808-nixos-system-photon-23.11pre516114.d680ded26da5/init loglevel=4 nvidia-drm.modeset=1
Aug 22 07:40:08 photon kernel: Kernel command line: initrd=\efi\nixos\rzgcs4y2b9mwb9a65g3c8xykdqq2lkby-initrd-linux-6.1.46-initrd.efi init=/nix/store/p4s0hpk3yr4gdl91rgrqvs4lj1j21808-nixos-system-photon-23.11pre516114.d680ded26da5/init loglevel=4 nvidia-drm.modeset=1
Aug 22 07:40:08 photon kernel: nvidia: loading out-of-tree module taints kernel.
Aug 22 07:40:08 photon kernel: nvidia: module license 'NVIDIA' taints kernel.
Aug 22 07:40:08 photon kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 244
Aug 22 07:40:08 photon kernel: nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
Aug 22 07:40:08 photon kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module  535.98  Tue Aug  1 21:42:05 UTC 2023
Aug 22 07:40:08 photon kernel: nvidia_uvm: module uses symbols nvUvmInterfaceDisableAccessCntr from proprietary module nvidia, inheriting taint.
Aug 22 07:40:08 photon kernel: nvidia-uvm: Loaded the UVM driver, major device number 242.
Aug 22 07:40:08 photon systemd-modules-load[796]: Inserted module 'nvidia_uvm'
Aug 22 07:40:08 photon systemd-modules-load[796]: Inserted module 'nvidia_modeset'
Aug 22 07:40:08 photon kernel: nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms  535.98  Tue Aug  1 21:40:14 UTC 2023
Aug 22 07:40:08 photon kernel: [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
Aug 22 07:40:09 photon (udev-worker)[829]: nvidia: Process '/nix/store/ir0j7zqlw9dc49grmwplppc7gh0s40yf-bash-5.2-p15/bin/bash -c 'mknod -m 666 /dev/nvidiactl c $(grep nvidia-frontend /proc/devices | cut -d \  -f 1) 255'' failed with exit code 1.
Aug 22 07:40:09 photon kernel: input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/0000:01:00.1/sound/card0/input23
Aug 22 07:40:09 photon kernel: input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/0000:01:00.1/sound/card0/input24
Aug 22 07:40:09 photon kernel: input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:01.1/0000:01:00.1/sound/card0/input25
Aug 22 07:40:09 photon kernel: input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:01.1/0000:01:00.1/sound/card0/input26
Aug 22 07:40:09 photon (udev-worker)[829]: nvidia: Process '/nix/store/ir0j7zqlw9dc49grmwplppc7gh0s40yf-bash-5.2-p15/bin/bash -c 'for i in $(cat /proc/driver/nvidia/gpus/*/information | grep Minor | cut -d \  -f 4); do mknod -m 666 /dev/nvidia${i} c $(grep nvidia-frontend /proc/devices | cut -d \  -f 1) ${i}; done'' failed with exit code 1.
Aug 22 07:40:10 photon systemd-modules-load[796]: Inserted module 'nvidia_drm'
Aug 22 07:40:10 photon kernel: [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 0
Aug 22 07:40:11 photon .gnome-shell-wr[1349]: Added device '/dev/dri/card0' (nvidia-drm) using atomic mode setting.
Aug 22 07:40:26 photon /nix/store/xdd83yx1qyis5drj8vi5g0p0nmx0513r-gdm-44.1/libexec/gdm-wayland-session[1775]: 00:00:00.005 [INFO] [backend/drm/backend.c:201] Initializing DRM backend for /dev/dri/card0 (nvidia-drm)
Aug 22 07:40:26 photon /nix/store/xdd83yx1qyis5drj8vi5g0p0nmx0513r-gdm-44.1/libexec/gdm-wayland-session[1775]: 00:00:00.011 [INFO] [render/egl.c:360] EGL vendor: NVIDIA
Aug 22 07:40:26 photon /nix/store/xdd83yx1qyis5drj8vi5g0p0nmx0513r-gdm-44.1/libexec/gdm-wayland-session[1775]: 00:00:00.011 [INFO] [render/egl.c:362] EGL driver name: nvidia
Aug 22 07:40:26 photon /nix/store/xdd83yx1qyis5drj8vi5g0p0nmx0513r-gdm-44.1/libexec/gdm-wayland-session[1775]: 00:00:00.038 [INFO] [render/gles2/renderer.c:844] Using OpenGL ES 3.2 NVIDIA 535.98
Aug 22 07:40:26 photon /nix/store/xdd83yx1qyis5drj8vi5g0p0nmx0513r-gdm-44.1/libexec/gdm-wayland-session[1775]: 00:00:00.038 [INFO] [render/gles2/renderer.c:845] GL vendor: NVIDIA Corporation
Aug 22 07:40:26 photon /nix/store/xdd83yx1qyis5drj8vi5g0p0nmx0513r-gdm-44.1/libexec/gdm-wayland-session[1775]: 00:00:00.038 [INFO] [render/gles2/renderer.c:846] GL renderer: NVIDIA GeForce RTX 4070/PCIe/SSE2
Aug 22 07:40:52 photon /nix/store/xdd83yx1qyis5drj8vi5g0p0nmx0513r-gdm-44.1/libexec/gdm-wayland-session[2088]: MESA-LOADER: failed to open nvidia-drm: /nix/store/afajk9q9cvsxcgw22vl9vh53h3bdgcpl-mesa-23.1.4/lib/gbm/nvidia-drm_gbm.so: cannot open shared object file: Permission denied (search paths /run/opengl-driver/lib/gbm:/nix/store/afajk9q9cvsxcgw22vl9vh53h3bdgcpl-mesa-23.1.4/lib/gbm, suffix _gbm)
Aug 22 07:40:52 photon /nix/store/xdd83yx1qyis5drj8vi5g0p0nmx0513r-gdm-44.1/libexec/gdm-wayland-session[2088]: MESA-LOADER: failed to open nvidia-drm: /run/opengl-driver/lib/dri/nvidia-drm_dri.so: cannot open shared object file: Permission denied (search paths /run/opengl-driver/lib/dri, suffix _dri)

Oh, I just remember, not sure though, that NVIDIA prime offloading doesn’t work on wayland. That may explain your issue. Could you try with a X window manager?

1 Like

Actually, that would only result in wayland applications not offloading, all the games running in Xwayland should work just fine. Either way, since the iGPU is turned off everything should be rendered on the dGPU anyway and prime is completely irrelevant. nvidia does sometimes not work well on wayland, but I’ve been using Hyprland with it for a while now (albeit with a slightly older GPU) and at least graphics acceleration is working properly, and I’m aware of 4070s driving Hyprland so I doubt it.

Note that journalctl --boot is a thing, by the way :slight_smile: I’d suggest rebooting, and then shortly after boot use journalctl --boot. Then there’s less logs to sift through, maybe you can share the full thing.

Hmm, iirc drm_dri is part of early modesetting (direct rendering manager, not digital rights management). Don’t know if this would be causing your symptoms though, this only happens in gdm. Have you tried launching Hyprland from a tty?

1 Like

Note that journalctl --boot is a thing, by the way :slight_smile: I’d suggest rebooting, and then shortly after boot use journalctl --boot. Then there’s less logs to sift through, maybe you can share the full thing.

Oh, I didn’t realize that, thanks for the advice! I disabled GDM and launched Hyprland, but still no luck in terms of GPU utilization. From that boot though, I saved the full outputs of journalctl --boot (I did this before launching Hyprland), which I’ve included in this pastebin since it’s quite large. The only thing that stands out to me from here are the lines about Failure creating named object, but again, I’m not sure whether those would the cause of my issue. There’s also some problem with loading rtw89 firmware, but I don’t think that that’s relevant since that seems to be a WiFi thing.

This performance issue seems to be really hard to pin down and it’s been driving up the wall, so thank you all for all the help, I really appreciate it!

Ok, long shot; I vaguely recall seeing ERR! for the fanspeed before, I think it was because my riser cable wasn’t properly seated. Can you make sure the card is 100% firm, fully in?

If that’s not it, try xorg, and also try running glxgears (from the mesa-demos package) with LIBGL_DEBUG=1 and MESA_DEBUG=1. Also nvidia-settings (doesn’t support wayland yet). If that doesn’t turn up anything I’m out of ideas.

1 Like

Darn, I was really hoping re-seating the GPU would fix it. It does seem like indeed the GPU fans stop spinning once the boot menu shows up. However, the seating of the GPU to my motherboard doesn’t seem to be my issue. Running glxgears with LIBGL_DEBUG=1 and MESA_DEBUG=1 doesn’t show anything new unfortunately. At this point, I’ll see if installing another OS will allow me to game, and if not, then I’ll try returning parts and redoing parts of my build :frowning:. Thank you all so much for your help! I really appreciate it all!

2 Likes

Just as an update for this in case anyone else comes upon this. The issue ended up being in my hardware. I had a faulty PCIe 4.0 riser cable, and forcing the BIOS to use PCIe 3.0 allows my graphics card to be utilized.

1 Like

See, I knew it was hardware! ;p

1 Like