Issues with Cinnamon / Plasa+xwayland in nixos 24.11 VM with Nvidia GPU passthrough or possibly something else?

e2g2024pc · February 11, 2025, 5:07pm

I am running nixos 24.11 in a VM. I was using KDE plasma, but wanted to try out Cinnamon.

I am using GPU passthrough of an nvidia card. I get errors on boot and/or just a black screen depending on what’s happening.

I changed my display settings from something like:

services.displayManager.sddm.enable = true;
services.desktopManager.plasma6.enable = true;

to

services.xserver.displayManager.lightdm.enable = true;
services.xserver.desktopManager.cinnamon.enable = true;

When I rebuild and reboot I never get the greeter. I saw another post suggested using ly:

services.displayManager = {
  enable = true;
  ly.enable = true;
};

I received a login greeter that was plaintext (assuming this is what ly normally looks like, have never used), but then after authentication I have a black screen.

When I finishing booting I see:

[FAILED] Failed to start Display Manager

× display-manager.service - Display Manager
     Loaded: loaded (/etc/systemd/system/display-manager.service; linked; preset: ignored)
     Active: failed (Result: exit-code) since Tue 2025-02-11 11:40:46 EST; 1min 50s ago
   Duration: 27ms
 Invocation: cb4c8ac0e0eb4825a8f1b69adbad1d02
    Process: 1536 ExecStartPre=/nix/store/yh8wysib97hms4m9nnppwhzl7j4h9jrp-unit-script-display-manager-pre-start/bin/display-manager-pre-start (code=exited, status=0/SUCCESS)
    Process: 1539 ExecStart=/nix/store/i2pd30k6wqs0ba9xnrwynx6imghyazcc-unit-script-display-manager-start/bin/display-manager-start (code=exited, status=1/FAILURE)
   Main PID: 1539 (code=exited, status=1/FAILURE)

Logs from display manager service:

Feb 11 11:49:39 [MY_HOST] systemd[1]: Starting Display Manager...
Feb 11 11:49:39 [MY_HOST] systemd[1]: Started Display Manager.
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Main process exited, code=exited, status=1/FAILURE
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Failed with result 'exit-code'.
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Triggering OnFailure= dependencies.
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Failed to enqueue OnFailure= job, ignoring: Unit plymouth-quit.service not found.
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Consumed 59ms CPU time, 15M memory peak, 10.4M read from disk.
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Scheduled restart job, restart counter is at 1.
Feb 11 11:49:39 [MY_HOST] systemd[1]: Starting Display Manager...
Feb 11 11:49:39 [MY_HOST] systemd[1]: Started Display Manager.
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Main process exited, code=exited, status=1/FAILURE
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Failed with result 'exit-code'.
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Triggering OnFailure= dependencies.
Feb 11 11:49:39 [MY_HOST] systemd[1]: display-manager.service: Failed to enqueue OnFailure= job, ignoring: Unit plymouth-quit.service not found.
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Scheduled restart job, restart counter is at 2.
Feb 11 11:49:40 [MY_HOST] systemd[1]: Starting Display Manager...
Feb 11 11:49:40 [MY_HOST] systemd[1]: Started Display Manager.
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Main process exited, code=exited, status=1/FAILURE
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Failed with result 'exit-code'.
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Triggering OnFailure= dependencies.
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Failed to enqueue OnFailure= job, ignoring: Unit plymouth-quit.service not found.
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Scheduled restart job, restart counter is at 3.
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Start request repeated too quickly.
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Failed with result 'exit-code'.
Feb 11 11:49:40 [MY_HOST] systemd[1]: Failed to start Display Manager.
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Triggering OnFailure= dependencies.
Feb 11 11:49:40 [MY_HOST] systemd[1]: display-manager.service: Failed to enqueue OnFailure= job, ignoring: Unit plymouth-quit.service not found.

Logs from /var/log/lightdm/lightdm.log:

[+0.00s] DEBUG: Logging to /var/log/lightdm/lightdm.log
[+0.00s] DEBUG: Starting Light Display Manager 1.32.0, UID=0 PID=1536
[+0.00s] DEBUG: Loading configuration dirs from /nix/store/5b40bwkn9rq7g1r5wf41qnnxz8cx08i6-desktops/share/lightdm/lightdm.conf.d
[+0.00s] DEBUG: Loading configuration dirs from /etc/xdg/lightdm/lightdm.conf.d
[+0.00s] DEBUG: Loading configuration from /etc/lightdm/lightdm.conf
[+0.00s] DEBUG: Registered seat module local
[+0.00s] DEBUG: Registered seat module xremote
[+0.00s] DEBUG: Using D-Bus name org.freedesktop.DisplayManager
[+0.00s] DEBUG: Using cross-namespace EXTERNAL authentication (this will deadlock if server is GDBus < 2.73.3)
[+0.00s] DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ‘gio-vfs’
[+0.00s] DEBUG: Monitoring logind for seats
[+0.00s] DEBUG: New seat added from logind: seat0
[+0.00s] DEBUG: Seat seat0: Loading properties from config section Seat:*
[+0.00s] DEBUG: Seat seat0 has property CanMultiSession=no
[+0.00s] DEBUG: Seat seat0: Starting
[+0.00s] DEBUG: Seat seat0: Creating greeter session
[+0.00s] DEBUG: Seat seat0: Creating display server of type x
[+0.00s] DEBUG: Using VT 7
[+0.00s] DEBUG: Seat seat0: Starting local X display on VT 7
[+0.00s] DEBUG: XServer 0: Logging to /var/log/lightdm/x-0.log
[+0.00s] DEBUG: XServer 0: Writing X server authority to /var/run/lightdm/root/:0
[+0.00s] DEBUG: XServer 0: Launching X Server
[+0.00s] DEBUG: Launching process 1543: /nix/store/9xjlnhbwr9pc10xf5jsm3c1jra23ccf0-xserver-wrapper :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
[+0.00s] DEBUG: XServer 0: Waiting for ready signal from X server :0
[+0.00s] DEBUG: Acquired bus name org.freedesktop.DisplayManager
[+0.00s] DEBUG: Registering seat with bus path /org/freedesktop/DisplayManager/Seat0
[+0.01s] DEBUG: Loading users from org.freedesktop.Accounts
[+0.01s] DEBUG: User /org/freedesktop/Accounts/User1000 added
[+0.05s] DEBUG: Process 1543 exited with return value 1
[+0.05s] DEBUG: XServer 0: X server stopped
[+0.05s] DEBUG: Releasing VT 7
[+0.05s] DEBUG: XServer 0: Removing X server authority /var/run/lightdm/root/:0
[+0.05s] DEBUG: Seat seat0: Display server stopped
[+0.05s] DEBUG: Seat seat0: Stopping session
[+0.05s] DEBUG: Seat seat0: Session stopped
[+0.05s] DEBUG: Seat seat0: Stopping display server, no sessions require it
[+0.05s] DEBUG: Seat seat0: Stopping; greeter display server failed to start
[+0.05s] DEBUG: Seat seat0: Stopping
[+0.05s] DEBUG: Seat seat0: Stopped
[+0.05s] DEBUG: Required seat has stopped
[+0.05s] DEBUG: Stopping display manager
[+0.05s] DEBUG: Display manager stopped
[+0.05s] DEBUG: Stopping daemon
[+0.05s] DEBUG: Exiting with return value 1

though one time through this i saw an error like this, but was not present in the current logs.

[+665.87s] WARNING: Error updating user /org/freedesktop/Accounts/User1000: GDBus.Error:org.freedesktop.DBus.Error.InvalidArgs: No such interface “org.freedesktop.DisplayManager.AccountsService”
[+671.65s] DEBUG: Got signal 15 from process 1
[+671.65s] DEBUG: Caught Terminated signal, shutting down
[+671.65s] DEBUG: Stopping display manager
[+671.65s] DEBUG: Seat seat0: Stopping
[+671.65s] DEBUG: Seat seat0: Stopping display server

I tried creating a fresh install from the Gnome gui installer and selecting Cinnamon and that would boot fine, cinnamon worked, etc.

I have all my nvidia settings in their own file, and I tried just not importing that settings, but it looks like with the pass through it defaulted back to nouveau which was throwing errors into my terminal all the time and it still didn’t boot.

Is this more likely just a Nvidia issue and not likely because I had KDE before? I saw some posts where people were setting different kernel versions, but have not yet tried that because I didn’t want to introduce more variables to consider. I also saw that some people said it worked, then a minor version broke?

Edit:
One additional note, I was unable to use xwayland with Plasma under my install with Nixos 24.11, on previous testing before I got passthrough running I was able to use xwayland with KDE Plasma. I’ve tried debugging or direct booting (e.g., startx), but I always get errors. Yes, I do have services.xserver.enable = true. So I am leaning maybe this is more of an Nvidia issue?

e2g2024pc · February 11, 2025, 5:20pm

I tried setting:

boot.kernelPackages = pkgs.linuxPackages_6_11;

and still got the warning and no greeter/login.

uname -r:

6.11.11

e2g2024pc · February 12, 2025, 7:51pm

Update: I took the working disk image for cinnamon made with a vm without passthrough using the installer defaults. I then mapped this image into my vm configured with passthrough. On boot, I got no errors, but received a black screen in the GUI tty, so more inclined to believe this is an NVIDIA issue then due to some package or cleanup conflict.

e2g2024pc · February 13, 2025, 6:23pm

Update:

In a VM with passthrough I tried to just install Linux Mint directly to see if this is a nix issue or not. When I installed Linux Mint it originally set me up in the VM with nouvaeu drivers, but i then used the GUI “Drivers” applet to install Nvidia proprietary drivers. Mint gave me the option to use “nvidia-driver-550” so I installed that rebooted… and it worked. I ran nvidia-smi and the nvidia gui settings to confirm it recognized the passed through card.

My conclusion from this is that there is so NixOS specific issue with the the nvidia drivers loading.

I’ve tried the following:

Tied to use Nixos 24.11 + cinnamon with Nvidia on bare metal instead of passthrough. It is the same issues. I could not even load i3 when i switched to using the Nvidia card directly with proprietary drivers.
I tried in my vm using the same version of the drivers as Mint was using successfully:

I tried both:

hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.legacy_550;

and manually specifying:

package = config.boot.kernelPackages.nvidiaPackages.mkDriver {
  version = "550.120";
  sha256_64bit = "sha256-gBkoJ0dTzM52JwmOoHjMNwcN2uBN46oIRZHAX8cDVpc=";
...

Using either of these, I would also not get the greeter.

I confirmed with nvidia-smi that it was in fact using 550.120

I saw in some posts and wikis to try things like

boot.initrd.kernelModules = [ "nvidia" ];
boot.extraModulePackages = [ config.boot.kernelPackages.nvidia_x11 ];

however, when I added this options and did nixos-rebuil switch I noticed it was then adding packages showing nvidia 565 instead of 550.

When I tried using nvidiaPackages.legacy_550; with the option services.xserver.videoDrivers = [ "nvidia" ]; it also tried to add in more current, unmatched stuff and gave me and error. When I manually specified the version, it allowed this setting.

At this point, I’m out of ideas. I have been using fresh VMs with the only things added were vim and then various nvidia settings. I am relatively certain I was using xwayland, etc with Nvidia when I first started playing with nix (24.05) a few months ago, because I only had the Nvidia card at the time and was specifically running into wayland issues.

I did try a fresh vm with 24.05 iso, but also got the same errors, so I am led to believe that something is not quite reproduceable here (why would it download 565 files if I am explicitly specifying 550?) or there is something I am missing.

Edit: I tried doing something similar to here: X refuses to start with Proprietary Nvidia Drivers - #6 by Gamer-Kold

but when I tried:

boot.extraModulePackages = [config.boot.kernelPackages.nvidiaPackages.legacy_550];

it told me 550 wasn’t an option, but 535 was. I tried legacy_535 and then also updated the driver to use “legacy_535”, but upon rebuild and reboot it would not boot X and nvidia-smi failed because it could not communicate with driver.

e2g2024pc · February 20, 2025, 6:17pm

Today I tried some new tests. I connected a second physical monitor to the the host (physical pc).

In the Linux Mint vm, I was able to run nvidia-xconfig. This produced /etc/X11/xorg.conf file that had an Nvidia entry and pointed to the PCI bus. After creating that file and rebooting, I saw the VM display load on the external monitor.

I next tried modifying around with the nixos guest vm I was using for testing. I noticed there was no nvidia-xconfig option in my nixos guest. I assume they want you to define the file in your configuration.nix by hand and that is why its not included (?). In testing this, I switched my services.xserver.videoDrivers from “nvidia” to “nvidia-x11”. I though perhaps this would provide the needed tools or special drivers? What did happen was that the external monitor connected to the Nvidia GPU HDMI turned on, but it still booted into spice. It seem that with the monitor connected (probably also a dummy plug). I could get x11 to boot in the spice graphics. The actual external monitor kept a black screen as well.

I next switch the services.xserver.videoDrivers back to “nvidia” and I got the same problems. I tried removing the Spice Graphics and the QXL video, hoping everything would just display on the external monitor. This was not the case and I didn’t see anything going to the serial monitor either.

This again suggests there is something different in the way nixos is handling this?

Update from later today:

On the host/hypervisor I noticed I had some settings from looking-glass client installed:

systemd.tmpfiles.rules = [
    "f /dev/shm/looking-glass 0660 [user] qemu-libvirtd -"
  ];

I had also installed the looking-glass client as a package. I think at some point I saw looking glass and thought maybe it could help, but there is no client for linux. After removing and rebuilding on the host, I noticed in another one of my test VMs I could use xwayland in plasma in the spice session. I don’t think I made any other changes on the host, so perhaps just setting up the /dev/shm and having the client installed (package, not like a config option) was somehow causing a problem in the guests… but only impacted x11, and only impacted nixos and not mint?

I made a fresh VM using UEFI. The VM boots into Cinnamon now after the initial install with no GPU passthrough connected. I then used the default settings to configure Nvidia drivers. After shutting down, attaching the passthrough devices, it now shows the NIXOS generations on both spice and on the external monitor. I then removed the spice graphics and set the video in virt-manager to none. After this boot, I now see the mint logon on the external monitor.

This would therefore appear to be working for the most part at this point, I still need to setup mouse/keyboard sharing via passthrough. I will note that on my mint VM I could still get to a tty on spice while the main display had the Cinnamon desktop. That would be beneficial for logging and other purposes.

I still want to do some more testing, but at this point it looks like the looking-glass config was interfering, though we would still expect some of the other issues and situations to be more consistent?