Yep, see the rule:
Specifically, /dev/dri/by-path/pci-0000:1:0.0 - that is subtly wrong; the bus and device are padded to have two characters. My new implementation actually does that correctly.
okay! I will substitute the rule for the new one and document here if any changes
I’ve substituted
services.udev.extraRules = let
nvidiaId = map (i: lib.toHexString (lib.toInt i)) (
lib.drop 1 (lib.splitString ":" config.hardware.nvidia.prime.nvidiaBusId)
);
domain = "0000";
bus = builtins.elemAt nvidiaId 0;
device = builtins.elemAt nvidiaId 1;
function = builtins.elemAt nvidiaId 2;
in ''
SUBSYSTEM=="drm", ENV{DEVTYPE}=="drm_minor", ENV{DEVLINKS}=="/dev/dri/by-path/pci-${domain}:${bus}:${device}.${function}", TAG+="mutter-device-preferred-primary"
'';
for
services.udev.packages =
let
pciPath =
xorgBusId:
let
components = lib.drop 1 (lib.splitString ":" xorgBusId);
toHex = i: lib.toLower (lib.toHexString (lib.toInt i));
domain = "0000"; # Apparently the domain is practically always set to 0000
bus = lib.fixedWidthString 2 "0" (toHex (builtins.elemAt components 0));
device = lib.fixedWidthString 2 "0" (toHex (builtins.elemAt components 1));
function = builtins.elemAt components 2; # The function is supposedly a decimal number
in
"dri/by-path/pci-${domain}:${bus}:${device}.${function}-card";
pCfg = config.hardware.nvidia.prime;
dgpuPath = pciPath pCfg.nvidiaBusId;
in
lib.singleton (
pkgs.writeTextDir "lib/udev/rules.d/61-gpu-offload.rules" ''
SYMLINK=="${dgpuPath}", TAG+="mutter-device-preferred-primary"
''
);
and after doing nixos-rebuild switch with no errors I restarted the system and, it seems that GDM is not working/loading (?) properly with this, booted into the system but I don’t get the GDM login screen even after a long time waiting.
Right, if that means you’re stuck, remember you can choose an older generation in the GRUB/systemd-boot menu (hold space on boot).
Could you switch to a tty (while GDM is broken, use ctrl+alt+F6) and give us the output of journalctl --boot -xe? You can pipe it to a file, reboot into a working generation and then paste it here.
The udevadm info stuff would also be nice to see in the new state, just to confirm card details.
so for the udevadmin info here are the results:
card1:
P: /devices/pci0000:00/0000:00:08.1/0000:05:00.0/drm/card1
M: card1
R: 1
J: c226:1
U: drm
T: drm_minor
D: c 226:1
N: dri/card1
L: 0
S: dri/by-path/pci-0000:05:00.0-card
E: DEVPATH=/devices/pci0000:00/0000:00:08.1/0000:05:00.0/drm/card1
E: DEVNAME=/dev/dri/card1
E: DEVTYPE=drm_minor
E: MAJOR=226
E: MINOR=1
E: SUBSYSTEM=drm
E: USEC_INITIALIZED=59977585
E: PATH=/nix/store/zz8r76ad9wmhav7myhk6nn244ch4lhlx-udev-path/bin:/nix/store/zz8r76ad9wmhav7myhk6nn244ch4lhlx-udev-path/sbin
E: ID_PATH=pci-0000:05:00.0
E: ID_PATH_TAG=pci-0000_05_00_0
E: ID_FOR_SEAT=drm-pci-0000_05_00_0
E: DEVLINKS=/dev/dri/by-path/pci-0000:05:00.0-card
E: TAGS=:seat:master-of-seat:uaccess:
E: CURRENT_TAGS=:seat:master-of-seat:uaccess:
P: /devices/pci0000:00/0000:00:01.1/0000:01:00.0/drm/card2
M: card2
R: 2
J: c226:2
U: drm
T: drm_minor
D: c 226:2
N: dri/card2
L: 0
S: dri/by-path/pci-0000:01:00.0-card
E: DEVPATH=/devices/pci0000:00/0000:00:01.1/0000:01:00.0/drm/card2
E: DEVNAME=/dev/dri/card2
E: DEVTYPE=drm_minor
E: MAJOR=226
E: MINOR=2
E: SUBSYSTEM=drm
E: USEC_INITIALIZED=59445287
E: PATH=/nix/store/zz8r76ad9wmhav7myhk6nn244ch4lhlx-udev-path/bin:/nix/store/zz8r76ad9wmhav7myhk6nn244ch4lhlx-udev-path/sbin
E: ID_PATH=pci-0000:01:00.0
E: ID_PATH_TAG=pci-0000_01_00_0
E: ID_FOR_SEAT=drm-pci-0000_01_00_0
E: DEVLINKS=/dev/dri/by-path/pci-0000:01:00.0-card
E: TAGS=:mutter-device-preferred-primary:seat:uaccess:master-of-seat:
E: CURRENT_TAGS=:mutter-device-preferred-primary:seat:uaccess:master-of-seat:
P: /devices/pci0000:00/0000:00:08.1/0000:05:00.0/drm/card1
M: card1
R: 1
J: c226:1
U: drm
T: drm_minor
D: c 226:1
N: dri/card1
L: 0
S: dri/by-path/pci-0000:05:00.0-card
E: DEVPATH=/devices/pci0000:00/0000:00:08.1/0000:05:00.0/drm/card1
E: DEVNAME=/dev/dri/card1
E: DEVTYPE=drm_minor
E: MAJOR=226
E: MINOR=1
E: SUBSYSTEM=drm
E: USEC_INITIALIZED=24266583
E: PATH=/nix/store/zz8r76ad9wmhav7myhk6nn244ch4lhlx-udev-path/bin:/nix/store/zz8r76ad9wmhav7myhk6nn244ch4lhlx-udev-path/sbin
E: ID_PATH=pci-0000:05:00.0
E: ID_PATH_TAG=pci-0000_05_00_0
E: ID_FOR_SEAT=drm-pci-0000_05_00_0
E: DEVLINKS=/dev/dri/by-path/pci-0000:05:00.0-card
E: TAGS=:uaccess:master-of-seat:seat:
E: CURRENT_TAGS=:uaccess:master-of-seat:seat:
card2:
P: /devices/pci0000:00/0000:00:01.1/0000:01:00.0/drm/card2
M: card2
R: 2
J: c226:2
U: drm
T: drm_minor
D: c 226:2
N: dri/card2
L: 0
S: dri/by-path/pci-0000:01:00.0-card
E: DEVPATH=/devices/pci0000:00/0000:00:01.1/0000:01:00.0/drm/card2
E: DEVNAME=/dev/dri/card2
E: DEVTYPE=drm_minor
E: MAJOR=226
E: MINOR=2
E: SUBSYSTEM=drm
E: USEC_INITIALIZED=23978206
E: PATH=/nix/store/zz8r76ad9wmhav7myhk6nn244ch4lhlx-udev-path/bin:/nix/store/zz8r76ad9wmhav7myhk6nn244ch4lhlx-udev-path/sbin
E: ID_PATH=pci-0000:01:00.0
E: ID_PATH_TAG=pci-0000_01_00_0
E: ID_FOR_SEAT=drm-pci-0000_01_00_0
E: DEVLINKS=/dev/dri/by-path/pci-0000:01:00.0-card
E: TAGS=:seat:master-of-seat:mutter-device-preferred-primary:uaccess:
E: CURRENT_TAGS=:seat:master-of-seat:mutter-device-preferred-primary:uaccess:
as for journalctl --boot -xe it’s here (had to upload it to an online paste else it would’ve surpassed the post character limit)
So for one, we can confirm it’s trying to use the nvidia GPU:
May 16 00:40:47 host .gnome-shell-wr[1509]: Thread 'KMS thread' will be using high priority scheduling May 16 00:40:47 host .gnome-shell-wr[1509]: Device '/dev/dri/card2' prefers shadow buffer May 16 00:40:47 host .gnome-shell-wr[1509]: Added device '/dev/dri/card2' (nvidia-drm) using atomic mode setting. May 16 00:40:47 host .gnome-shell-wr[1509]: Device '/dev/dri/card1' prefers shadow buffer May 16 00:40:47 host .gnome-shell-wr[1509]: Added device '/dev/dri/card1' (amdgpu) using atomic mode setting. May 16 00:40:47 host .gnome-shell-wr[1509]: Created gbm renderer for '/dev/dri/card2' May 16 00:40:47 host .gnome-shell-wr[1509]: Created gbm renderer for '/dev/dri/card1' May 16 00:40:47 host .gnome-shell-wr[1509]: GPU /dev/dri/card2 selected primary given udev rule May 16 00:40:47 host .gnome-shell-wr[1509]: Obtained a high priority EGL context
Not really surprising, because why else would it stop working? Still nice for a sanity check. Your udevadm output looks good, too.
Next up, why is it not working?
May 16 00:40:48 host .gnome-shell-wr[1509]: Failed to create view for Built-in display on eDP-1: Failed to allocate onscreen framebuffer for /dev/dri/card1: Failed to allocate surface: Function not implemented May 16 00:40:48 host .gnome-shell-wr[1509]: Failed to create view for Built-in display on eDP-1: Failed to allocate onscreen framebuffer for /dev/dri/card1: Failed to allocate surface: Function not implemented
Looks to be an upstream issue: With Nvidia as primary, screen attached to secondary (integrated) GPU does not light up (#3918) · Issues · GNOME / mutter · GitLab. Given it’s not been entirely solved I think you’re out of luck for using NVIDIA as your primary GPU on GNOME for now; depending on how release cadences fall, maybe until November.
I’d suggest getting familiar with the nvidia-offload command for now, the performance difference really is negligible, and even if you don’t actually care, your battery’s lifespan will be better if it gets taxed less. All you need to do is launch the applications you want to run on the dGPU with nvidia-offload.
I see I see, so I guess I will be using the first set of rules posted here then and wait to see when this is going to be solved, I guess this is truly an NVIDIA moment.
thanks a lot for the help tho! really appreciate it and I hope this post can also help more people in the future
1 Like
FWIW, my nvidia module is now nearly complete; this thread helped quite a bit with it, actually. I’ll factor it out of my personal dotfiles in the near future, but if you’d like to see what I consider an idiomatic and correct nvidia configuration, there you go.
No support for using nvidia as the primary GPU yet, but I’m considering adding that as a specialization.
1 Like
I see I see, I’m glad that this could be of help in a way, if you ever manage to make NVIDIA as the primary GPU usable feel free to post here
love your work man, you help me a lot in the past as well. eagerly waiting for your module to come out. 
1 Like
I feel I should also add this to the thread btw for future people reading it:
in case of a Steam game not using the dGPU but the iGPU instead, you can set the following on game launch options for it to do so: __NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia %command%
big thanks to reddit user u/Holzkohlen at r/linux_gaming for that!
I will also probably do testings on emulators (mainly the PS3 one) to see if they detect the dGPU or not and what workarounds can be done
There are actually a few more variables you should set: nixpkgs/nixos/modules/hardware/video/nvidia.nix at 72841a4a8761d1aed92ef6169a636872c986c76d · NixOS/nixpkgs · GitHub
On NixOS, you can enable hardware.nvidia.prime.offload.enableOffloadCmd to get the nvidia-offload binary I’ve been talking about. You can replace your steam command with it:
nvidia-offload %command%
In fact, any program you launch with nvidia-offload should be offloaded, since that’s just fundamentally how the driver works.
This includes emulators; if they launch additional windows and don’t have steam-style launch command config, you can just launch the emulator with nvidia-offload - any child processes will inherit the environment and also be offloaded.
1 Like
oh I see, good to know thanks a lot!!