NVidia offload vs. sync vs

Hello all,

I’ve been using NixOS for a little while now, loving it! My main laptop (Lenovo Thinkpad P50) has both built-in Intel and additional NVidia hardware. The latter is also needed for anything with the HDMI port, which would be handy.

I’m starting with what I have read here: https://nixos.wiki/wiki/Nvidia, and would prefer to use offload mode.

The problem(s) I’m seeing is that while I have the open-source driver disabled, it’s still showing Nouveau in the Xorg.0.log for DRI, and X fails to start. Once though, without changing anything, it did work! I downloaded glxgears, glmark2, and both ran fine. And then I rebooted, and X was toast again.

If I disable offload, I can run sync mode; X starts, but the glxgears and glmark programs fail.

One of the other questions that came to mind is that the wiki entry above shows hardware.nvidia.open = true. Does this make sense when it’s otherwise describing use of the proprietary driver? None of what I am trying to do is performance-critical, but in any case, trying that option as well didn’t seem to help.

From lspci:

00:02.0 VGA compatible controller: Intel Corporation HD Graphics 530 (rev 06)
01:00.0 VGA compatible controller: NVIDIA Corporation GM107GLM [Quadro M1000M] (rev a2)

NixOS channels:

nixos nixos-23.05 release nixos-23.05.1272.ecb441f2206
nixos-hardware https://github.com/NixOS/nixos-hardware/archive/master.tar.gz

Kernel: Linux peefiddy 6.1.34 #1-NixOS SMP PREEMPT_DYNAMIC Wed Jun 14 09:15:34 UTC 2023 x86_64 GNU/Linux

From /etc/nixos/configuration.nix (in offload mode):

NVidia driver support from Nvidia - NixOS Wiki

Make sure opengl is enabled

hardware.opengl = {
enable = true;
driSupport = true;
driSupport32Bit = true;
};

Tell Xorg to use the nvidia driver

services.xserver.videoDrivers = [“nvidia”];
hardware.nvidia.prime = {
offload = {
enable = true;
enableOffloadCmd = true;
};
# sync.enable = true;
intelBusId = “PCI:1:2:0”;
nvidiaBusId = “PCI:1:0:0”;
};

hardware.nvidia = {

# Modesetting is needed for most wayland compositors
modesetting.enable = false;

# Use the open source version of the kernel module
open = false;

# Enable the nvidia settings menu
nvidiaSettings = true;

# Optionally, you may need to select the appropriate driver version for your specific GPU.
package = config.boot.kernelPackages.nvidiaPackages.stable;

};

The relevant, latter part of my Xorg.0.log:

[ 108.328] (==) modeset(0): Backing store enabled
[ 108.328] (==) modeset(0): Silken mouse enabled
[ 108.408] (II) modeset(0): Initializing kms color map for depth 24, 8 bpc.
[ 108.408] (==) modeset(0): DPMS enabled
[ 108.408] (II) modeset(0): [DRI2] Setup complete
[ 108.408] (II) modeset(0): [DRI2] DRI driver: iris
[ 108.408] (II) modeset(0): [DRI2] VDPAU driver: va_gl
[ 108.503] (==) modeset(G0): Backing store enabled
[ 108.503] (==) modeset(G0): Silken mouse enabled
[ 108.503] (II) modeset(G0): Initializing kms color map for depth 24, 8 bpc.
[ 108.503] (==) modeset(G0): DPMS enabled
[ 108.504] (II) modeset(G0): [DRI2] Setup complete
[ 108.504] (II) modeset(G0): [DRI2] DRI driver: nouveau
[ 108.504] (II) modeset(G0): [DRI2] VDPAU driver: nouveau
[ 108.504] (EE)
[ 108.504] (EE) Backtrace:
[ 108.505] (EE) 0: /nix/store/y9l9mbf313idyqck1zbv6s72xj1a4phy-xorg-server-21.1.8/bin/X (OsSigHandler+0x29) [0x5b63a9]
[ 108.505] (EE) 1: /nix/store/dg8mpqqykmw9c7l0bgzzb5znkymlbfjw-glibc-2.37-8/lib/libc.so.6 (__sigaction+0x50) [0x7fd0de67cd60]
[ 108.506] (EE) 2: /nix/store/y9l9mbf313idyqck1zbv6s72xj1a4phy-xorg-server-21.1.8/bin/X (InitOutput+0x4b4) [0x487fb4]
[ 108.506] (EE) 3: /nix/store/y9l9mbf313idyqck1zbv6s72xj1a4phy-xorg-server-21.1.8/bin/X (dix_main+0x190) [0x449160]
[ 108.506] (EE) 4: /nix/store/dg8mpqqykmw9c7l0bgzzb5znkymlbfjw-glibc-2.37-8/lib/libc.so.6 (__libc_start_call_main+0x7e) [0x7fd0de667ace]
[ 108.507] (EE) 5: /nix/store/dg8mpqqykmw9c7l0bgzzb5znkymlbfjw-glibc-2.37-8/lib/libc.so.6 (__libc_start_main+0x89) [0x7fd0de667b89]
[ 108.507] (EE) 6: /nix/store/y9l9mbf313idyqck1zbv6s72xj1a4phy-xorg-server-21.1.8/bin/X (_start+0x25) [0x432785]
[ 108.508] (EE)
[ 108.508] (EE) Segmentation fault at address 0x10
[ 108.508] (EE)
Fatal server error:
[ 108.508] (EE) Caught signal 11 (Segmentation fault). Server aborting
[ 108.508] (EE)
[ 108.508] (EE)
Please consult the The X.Org Foundation support
at http://wiki.x.org
for help.
[ 108.508] (EE) Please also check the log file at “/var/log/Xorg.0.log” for additional information.
[ 108.508] (EE)
[ 108.560] (EE) Server terminated with error (1). Closing log file.

Your graphics card model? It may not be in the latest supported list.

Check your model to try?

"nvidiaLegacy390"
"nvidiaLegacy340"
"nvidiaLegacy304"

Thanks @Stzx. It’s a Quadro M1000M, which is listed as supported by (for example) the 470 version at Linux x64 (AMD64/EM64T) Display Driver | 470.63.01 | Linux 64-bit | NVIDIA.

For one tantalizing short while, it loaded fine, and was happily chugging through GLMark2 benchmarks in offload mode. So clearly it can do it, but it hasn’t been reliably loading.

Cheers,
Tim

Check your bus ID, (replies should not be less than 20 words …)

Oh, FFS. Thanks, that’s one footgun taken care of. I’m more curious how it worked the one time it did.

Regarding driver versions, mine (Quadro M1000M) is supported in current packages, but does not have the GSP functionality that it looks for, so loading X was still failing.

The nvidia driver was loading, but needs a parameter set in my case, e.g. manually:
sudo modprobe nvidia NVreg_OpenRmEnableUnsupportedGpus=1

I did add the following which should set things correctly:

boot.extraModprobeConfig = ‘’
options nvidia NVreg_OpenRmEnableUnsupportedGpus=1
‘’;

While I find that I have the option set in /etc/modprobe.d/nixos.conf, it still seems to require that I manually rmmod the modules that got loaded, and reload nvidia with this option set. Then I can run sddm manually, and it works.

For what it’s worth, I’ve got hardware.nvidia.open = false.

The reason is that while X wasn’t crashing entirely, the open drivers don’t support my device, with this message:

[10016.814706] nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[10016.814726] NVRM: The NVIDIA GPU 0000:01:00.0 (PCI ID: 10de:13b1)
NVRM: installed in this system is not supported by open
NVRM: nvidia.ko because it does not include the required GPU
NVRM: System Processor (GSP).
NVRM: Please see the ‘Open Linux Kernel Modules’ and ‘GSP
NVRM: Firmware’ sections in the driver README, available on
NVRM: the Linux graphics driver download page at
NVRM: www.nvidia.com.
[10016.814765] nvidia: probe of 0000:01:00.0 failed with error -1
[10016.814781] NVRM: The NVIDIA probe routine failed for 1 device(s).
[10016.814783] NVRM: None of the NVIDIA devices were initialized.
[10016.815039] nvidia-nvlink: Unregistered Nvlink Core, major device number 239
[10017.172544] nvidia-nvlink: Nvlink Core is being initialized, major device number 239

OK, confirmed that the boot.extraModprobeConfig part does not seem to be affecting things in time for the displayManager (in this case, SDDM) to start. (E.g., I reboot, still need to manually rmmod the nvidia* modules, kill the lingering sddm process, and then run systemctl start display-manager).

That should likely be a separate issue, since the basic configuration aspect I ran into has been addressed.