Passing through integrated graphics to libvirt VM?

Help
1

I looked at a few articles / pages listed below, mostly following the Arch wiki guide. I have an Nvidia GPU which I’m using for Nix, and Intel integrated graphics which I’m trying to passthrough to a Windows VM. I connected my motherboard HDMI to my monitor and it shows up as a 2nd monitor for Nix. I tried adding the Intel graphics as a PCIE device in the VM but it then nothing shows up from the HDMI port on my monitor. When I run the bash script under https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#:~:text=Ensuring_that_the_groups_are_valid, I get the following:

  • 00:02.0 Display controller [0380]: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] [8086:3e98] ( rev 02 )
  • 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU116 [GeForce GTX 1660 SUPER] [10de:21c4] (rev a1)

Any ideas for what to do? Would could I blacklist the Intel graphics from Nix? Is the Intel graphics supposed to be called “Display Controller” and not "VGA Compatible controller?

  1. https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF
  2. https://alexbakker.me/post/nixos-pci-passthrough-qemu-vfio.html
  3. https://astrid.tech/2022/09/22/0/nixos-gpu-vfio/

My config:

      programs.virt-manager.enable = true;
      virtualisation.spiceUSBRedirection.enable = true;
    
      virtualisation.libvirtd = {
        enable = true;
        qemu = {
          package = pkgs.qemu_kvm;
          runAsRoot = true;
          swtpm.enable = true;
          ovmf = {
            enable = true;
            packages = [
              (pkgs.OVMF.override {
                secureBoot = true;
                tpmSupport = true;
              })
            ];
          };
        };
      };
    
    
        kernelModules = [
          "uinput"
    
          "vfio_pci"
          "vfio"
          "vfio_iommu_type1"
        ];
        kernelParams = [
    
          "intel_iommu=on"
          "vfio-pci.ids=8086:3e98"
          "iommu=pt"
        ];
    
      boot.extraModulePackages = [ config.boot.kernelPackages.kvmfr ];
      boot.extraModprobeConfig = ''
        options kvmfr static_size_mb=128
      '';
      boot.initrd.kernelModules = [
        "kvmfr"
      ];
      services.udev.extraRules = ''
        SUBSYSTEM=="kvmfr", OWNER="${config.users.users.yousuf.name}", GROUP="qemu-libvirtd", MODE="0600"
      '';
    
        virtualisation.libvirtd.qemu.verbatimConfig = ''
            cgroup_device_acl = [
                "/dev/null", "/dev/full", "/dev/zero",
                "/dev/random", "/dev/urandom",
                "/dev/ptmx", "/dev/kvm",
                "/dev/userfaultfd", "/dev/kvmfr0"
            ]
          '';
    
      networking.firewall.trustedInterfaces = [ "virbr0" ];
      systemd.services.libvirt-default-network = {
        description = "Start libvirt default network";
        after = [ "libvirtd.service" ];
        wantedBy = [ "multi-user.target" ];
        serviceConfig = {
          Type = "oneshot";
          RemainAfterExit = true;
          ExecStart = "${pkgs.libvirt}/bin/virsh net-start default";
          ExecStop = "${pkgs.libvirt}/bin/virsh net-destroy default";
          User = "root";
        };
      };
2

After doing some more research (more than it should’ve taken), it looks like the graphics on my i9 9900k can’t just be passed through like that. They showed up in Windows device manager but showed error code 43. I might need a bios like this one: GitHub - patmagauran/i915ovmfPkg: VBIOS for Intel GPU Passthrough, or use Intel GV-T (which I’m not going to).

3

I passthru a lot of PCIe devices to VMs but I haven’t passed through a GPU. In my learning process, one thing appeared to stand out and that was there were two parts to passthru.

  1. Optionally disable the host from using a device using the vfio-pci stub.
  2. Passthru a PCIe device.

Everything I read indicated that the first step was particularly important when it came to GPUs. Your config above doesn’t seem to show this step.

This example might help…

I am not sure this will address the issues of passing through the iGPU but it may help you do it the other way around. It looks like your dGPU is a dedicated card (not muxed like in a laptop), so I’m intrigued by the idea that you can’t passthru the iGPU. Be really interested to hear if you do get this to work. Good luck.

4

Do you mean extraModprobeConfig? I tried that but again, the GPU showed up in device manager but showed error 43.

5

I believe the boot.extraModprobeConfig and boot.initrd.preDeviceCommands methods above may achieve the same thing but I do both.

The only other thing is to ensure that you are passing through an entire IOMMU group. There is a bash command that will show you the IOMMU groups’

shopt -s nullglob
for d in /sys/kernel/iommu_groups/*/devices/*;do  n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'iommu GROUP %s ' "$n"; lspci -nns "${d##*/}"; done;

This output from a laptop shows the dGPU is in IOMMU group 15 and there is also an audio device. Both need to be passed through. The iGPU is group 2 and is on it’s own.

iommu GROUP 15 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104M [GeForce RTX 3070 Mobile / Max-Q] [10de:249d] (rev a1)
iommu GROUP 15 01:00.1 Audio device [0403]: NVIDIA Corporation GA104 High Definition Audio Controller [10de:228b] (rev a1)
iommu GROUP 2 00:02.0 VGA compatible controller [0300]: Intel Corporation TigerLake-H GT1 [UHD Graphics] [8086:9a60] (rev 01)

(Windows) Device Manager error 43 seems to cover a range of issues including drivers. Are you emulating a different CPU perhaps or does you libvirt config include a QXL or virtio video device too? I presume spice displays won’t work either. Also is your iGPU being powered down by your UEFI firmware because you have a dGPU? These are just guesses.

6

The GPU is the only one in the IOMMU group. I had a regular video device to see windows and device manager.

Also is your iGPU being powered down by your UEFI firmware because you have a dGPU? These are just guesses.

Would the GPU show up in device manager at all if that was the case? Idk.

7

If you remove the ‘regular’ video device. Do you get any output on the attached monitor when you boot? BIOS/UEFI firmware pressing F2? Boot splash?

8

No, right after I start the VM up I switch my monitor input to the one connected to the motherboard for the iGPU but it shows nothing.

9

I had this working previously on Intel HD 4000 which was passed to macOS and Linux VMs but I never tried Windows. Can you try to passthru it to a Linux VM first? If nothing happens then try booting the VM with pci=nocrs (added to cmdline of the guest)
If Linux works, then try to passthrough it to Windows and try to install Intel GPU drivers. If installer will fail because of error 43, then try to unpack it somehow and install from .inf

Optionally disable the host from using a device using the vfio-pci stub.
Passthru a PCIe device.

Everything I read indicated that the first step was particularly important when it came to GPUs. Your config above doesn’t seem to show this step.

I think it depends on GPUs, on my RX460 there was no difference, it’s just that everything graphics-related would crash before VM started.
Newer GPUs have some DRM-related (afaik) stuff that requires the whole GPU to power off before it can be used in a different environment (eg. Linux → Windows) - using vfio-pci stub makes sure that VM gets the GPU untouched. There’s also vendor-reset module which can reset some of those GPUs properly without having to completely reboot the host.

1 Like
10

Disregard what I said earlier about forgetting about it. I’ve been thinking, why would I upgrade from a 9900k if I’m not bottlenecked by it? If I can’t get iGPU passthrough to work I actually already have a spare GPU and PCIE slot for it.

11

@nat @fsbof I want to try getting GVT-g work instead: Getting Intel GVT-g (GPU virtualization) working?.

12

I’m really intrigued by this but I don’t really have a setup I can test with at the moment sorry. I will follow your progress if you are happy to share. :pray:

13

I gave up actually, I sold the CPU (i9 9900K) as well as the Motherboard and RAM and gonna get new ones. Already paid more than I should’ve for ram, I’m gonna get a Ryzen 9600x.

14

Well, good luck. Hope the new setup works out better for you.