Issue running nixosTest in nested virtualization case on Hyper-V

My setup runs nixos as an hyper-v guest, and I’m having issues with running any type L2 guest. This is especially annoying for nixosTest instances.

[ Hardware: Physical CPU (VT-x) ]
         │
[ L0 Host: Windows 11 + Hyper-V ]
         │
[ L1 Guest VM: NixOS (Hyper-V VM) ]
         │
[ L2 Guest VMs: nixosTest framework (QEMU using /dev/kvm) ]

On my computer the L2 guest hangs during boot.

I’m starting to feel like this setup is not supposed to “just work”. And this is going into territory way above my head. It looks like a (spin)lock where the vCPU just doesn’t advance, no errors or warnings thrown by the L2 guest kernel. Hopefully someone more experienced knows what is going on here :slight_smile:

I took a trace from L1 during L2 boot, which is summarized by claude like this;

The ftrace capture at the hang point showed ~150,000 vmexits/sec sustained with only 1 HLT across 401,042 events — a genuine livelock (continuous execution that never completes), not a stuck or deadlocked vCPU.
Exit reasons were dominated by IO_INSTRUCTION (87,599, mostly serial-console LSR/THR polling and PCI config-space access) and EPT_VIOLATION+EPT_MISCONFIG (64,880 combined), while MSR_READ/MSR_WRITE were negligible (103/57) — meaning the guest's local APIC was still in legacy xAPIC (memory-mapped) mode, so every LAPIC touch (EOI, IPI, TPR) trapped through the unaccelerated MMIO/EPT path rather than a cheap MSR trap.
That EPT/MMIO-trap dominance directly corroborates this host's flexpriority=N/enable_apicv=N (confirmed via kvm_intel module parameters and /proc/cpuinfo's vmx flags) — Hyper-V's nested-VMX exposure omits the whole APICv/TPR-shadow secondary-control group, forcing every interrupt-related APIC access into software emulation.
Trace generator script
#!/usr/bin/env bash
set -x
cd ~/a-box
TR=/sys/kernel/debug/tracing

echo 0 > $TR/tracing_on
echo > $TR/trace
echo 8192 > $TR/buffer_size_kb

for ev in kvm_exit kvm_entry kvm_pio kvm_msr; do
  [ -e "$TR/events/kvm/$ev/enable" ] && echo 1 > "$TR/events/kvm/$ev/enable"
done
echo 1 > $TR/tracing_on

timeout 20 nix-build --no-out-link -E 'let self = import ./llm-host.nix {}; in self.platformMemoryPostureTest' || true

echo 0 > $TR/tracing_on
cp $TR/trace /tmp/kvm-trace.log
chmod 644 /tmp/kvm-trace.log

for ev in kvm_exit kvm_entry kvm_pio kvm_msr; do
  [ -e "$TR/events/kvm/$ev/enable" ] && echo 0 > "$TR/events/kvm/$ev/enable"
done

wc -l /tmp/kvm-trace.log
echo "=== exit_reason histogram ==="
grep -oP 'exit_reason \K[0-9]+' /tmp/kvm-trace.log | sort | uniq -c | sort -rn | head -20
Full L2 boot log
starting vm
QEMU running (pid 45)
 Disk image does not exist, creating the virtualisation disk image...
waiting for unit multi-user.target
 Formatting '/build/vm-state-machine/tmp.HUe0nKHqxz', fmt=raw size=1073741824
 mke2fs 1.47.4 (6-Mar-2025)
waiting for the VM to finish booting
 Discarding device blocks: done
 Creating filesystem with 262144 4k blocks and 65536 inodes
 Filesystem UUID: 6bd126b2-e6d3-49f1-b081-fd7c4791adae
 Superblock backups stored on blocks:
       32768, 98304, 163840, 229376

 Allocating group tables: done
 Writing inode tables: done
 Creating journal (8192 blocks): done
 Writing superblocks and filesystem accounting information: done

 Virtualisation disk image created.
 Creating Nix store image...
 Created Nix store image.
 Starting virtiofs daemons...
 cSeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)


 iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+2EFCC670+2EF2C670 CA00
 Press Ctrl-B to configure iPXE (PCI 00:02.0)...


 iPXE (http://ipxe.org) 00:06.0 CB00 PCI2.10 PnP PMM 2EFCC670 2EF2C670 CB00
 Press Ctrl-B to configure iPXE (PCI 00:06.0)...


 Booting from ROM...
 Probing EDD (edd=off to disable)... ock[    0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026
0.000000 Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b95a4vfflrim7zfhzdqygpls1cd64c2s-nixos-system-machine-test/init regInfo=/nix/store/g2pm9xlj92il0clxlvsr90xlnx25n4qq-closure-info/registration console=ttyS0,115200n8 console=tty0
0.000000 BIOS-provided physical RAM map:
0.000000 BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
0.000000 BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
0.000000 BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
0.000000 BIOS-e820: [mem 0x0000000000100000-0x000000002ffd4fff] usable
0.000000 BIOS-e820: [mem 0x000000002ffd5000-0x000000002fffffff] reserved
0.000000 BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved
0.000000 BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
0.000000 BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
0.000000 BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
0.000000 NX (Execute Disable) protection: active
0.000000 APIC: Static calls initialized
0.000000 SMBIOS 2.8 present.
0.000000 DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
0.000000 DMI: Memory slots populated: 1/1
0.000000 Hypervisor detected: KVM
0.000000 last_pfn = 0x2ffd5 max_arch_pfn = 0x400000000
0.000000 kvm-clock: Using msrs 4b564d01 and 4b564d00
0.000001 kvm-clock: using sched offset of 4713860643233 cycles
0.000002 clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
0.000004 tsc: Detected 3700.000 MHz processor
0.000688 last_pfn = 0x2ffd5 max_arch_pfn = 0x400000000
0.000789 MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs
0.000794 x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT
0.006078 found SMP MP-table at [mem 0x000f5460-0x000f546f]
0.006089 Using GB pages for direct mapping
0.006164 RAMDISK: [mem 0x2e371000-0x2ffcffff]
0.006176 ACPI: Early table checksum verification disabled
0.006178 ACPI: RSDP 0x00000000000F5280 000014 (v00 BOCHS )
0.006184 ACPI: RSDT 0x000000002FFE2400 000038 (v01 BOCHS  BXPC     00000001 BXPC 00000001)
0.006188 ACPI: FACP 0x000000002FFE21F8 0000F4 (v03 BOCHS  BXPC     00000001 BXPC 00000001)
0.006198 ACPI: DSDT 0x000000002FFE0040 0021B8 (v01 BOCHS  BXPC     00000001 BXPC 00000001)
0.006200 ACPI: FACS 0x000000002FFE0000 000040
0.006203 ACPI: APIC 0x000000002FFE22EC 000078 (v03 BOCHS  BXPC     00000001 BXPC 00000001)
0.006215 ACPI: HPET 0x000000002FFE2364 000038 (v01 BOCHS  BXPC     00000001 BXPC 00000001)
0.006217 ACPI: MCFG 0x000000002FFE239C 00003C (v01 BOCHS  BXPC     00000001 BXPC 00000001)
0.006219 ACPI: WAET 0x000000002FFE23D8 000028 (v01 BOCHS  BXPC     00000001 BXPC 00000001)
0.006221 ACPI: Reserving FACP table memory at [mem 0x2ffe21f8-0x2ffe22eb]
0.006222 ACPI: Reserving DSDT table memory at [mem 0x2ffe0040-0x2ffe21f7]
0.006223 ACPI: Reserving FACS table memory at [mem 0x2ffe0000-0x2ffe003f]
0.006223 ACPI: Reserving APIC table memory at [mem 0x2ffe22ec-0x2ffe2363]
0.006223 ACPI: Reserving HPET table memory at [mem 0x2ffe2364-0x2ffe239b]
0.006224 ACPI: Reserving MCFG table memory at [mem 0x2ffe239c-0x2ffe23d7]
0.006224 ACPI: Reserving WAET table memory at [mem 0x2ffe23d8-0x2ffe23ff]
0.006931 No NUMA configuration found
0.006932 Faking a node at [mem 0x0000000000000000-0x000000002ffd4fff]
0.006934 NODE_DATA(0) allocated [mem 0x2e36ba80-0x2e370fff]
0.007164 Zone ranges:
0.007165   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
0.007166   DMA32    [mem 0x0000000001000000-0x000000002ffd4fff]
0.007167   Normal   empty
0.007168   Device   empty
0.007168 Movable zone start for each node
0.007168 Early memory node ranges
0.007169   node   0: [mem 0x0000000000001000-0x000000000009efff]
0.007169   node   0: [mem 0x0000000000100000-0x000000002ffd4fff]
0.007170 Initmem setup node 0 [mem 0x0000000000001000-0x000000002ffd4fff]
0.007201 On node 0, zone DMA: 1 pages in unavailable ranges
0.007605 On node 0, zone DMA: 97 pages in unavailable ranges
0.028838 On node 0, zone DMA32: 43 pages in unavailable ranges
0.030345 ACPI: PM-Timer IO Port: 0x608
0.030364 ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
0.030519 IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
0.030537 ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
0.030539 ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
0.030539 ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
0.030540 ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
0.030541 ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
0.030544 ACPI: Using ACPI (MADT) for SMP configuration information
0.030544 ACPI: HPET id: 0x8086a201 base: 0xfed00000
0.030547 TSC deadline timer available
0.030550 CPU topo: Max. logical packages:   1
0.030551 CPU topo: Max. logical dies:       1
0.030551 CPU topo: Max. dies per package:   1
0.030554 CPU topo: Max. threads per core:   1
0.030554 CPU topo: Num. cores per package:     1
0.030555 CPU topo: Num. threads per package:   1
0.030555 CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs
0.030577 kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()
0.030625 PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
0.030626 PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]
0.030627 [mem 0x30000000-0xafffffff] available for PCI devices
0.030628 Booting paravirtualized kernel on KVM
0.030630 clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
0.034401 setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
0.037941 percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152
0.038003 kvm-guest: PV spinlocks disabled, single CPU
0.038004 Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b95a4vfflrim7zfhzdqygpls1cd64c2s-nixos-system-machine-test/init regInfo=/nix/store/g2pm9xlj92il0clxlvsr90xlnx25n4qq-closure-info/registration console=ttyS0,115200n8 console=tty0
0.038083 Unknown kernel command line parameters "regInfo=/nix/store/g2pm9xlj92il0clxlvsr90xlnx25n4qq-closure-info/registration", will be passed to user space.
0.038097 random: crng init done
0.038098 printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes
0.039850 Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
0.039869 Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
0.039928 Fallback order for Node 0: 0
0.039930 Built 1 zonelists, mobility grouping on.  Total pages: 196467
0.039931 Policy zone: DMA32
0.044079 mem auto-init: stack:all(zero), heap alloc:on, heap free:off
0.046856 SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
0.049583 allocated 1572864 bytes of page_ext
0.056763 ftrace: allocating 48787 entries in 192 pages
0.056764 ftrace: allocated 192 pages with 2 groups
0.057733 Dynamic Preempt: lazy
0.057971 rcu: Preemptible hierarchical RCU implementation.
0.057971 rcu:   RCU event tracing is enabled.
0.057972 rcu:   RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.
0.057973        Trampoline variant of Tasks RCU enabled.
0.057973        Rude variant of Tasks RCU enabled.
0.057973        Tracing variant of Tasks RCU enabled.
0.057974 rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
0.057974 rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
0.058063 RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.
0.058065 RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.
0.058066 RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.
0.062896 NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16
0.063300 rcu: srcu_init: Setting srcu_struct sizes based on contention.
0.063309 clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
0.063400 kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)
0.073199 Console: colour VGA+ 80x25
0.073201 printk: legacy console [tty0] enabled
0.139470 printk: legacy console [ttyS0] enabled
0.509611 ACPI: Core revision 20250807
0.512232 clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
0.517316 APIC: Switch to symmetric I/O mode setup
0.521222 x2apic enabled
0.526997 APIC: Switched APIC routing to: physical x2apic
0.537740 ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
0.541304 clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x6aaaabc71c7, max_idle_ns: 881590412124 ns
0.546822 Calibrating delay loop (skipped) preset value.. 7400.00 BogoMIPS (lpj=3700000)
0.549524 x86/cpu: User Mode Instruction Prevention (UMIP) activated
0.551273 Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
0.552822 Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
0.555860 mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto
0.556822 Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
0.558822 SRBDS: Unknown: Dependent on hypervisor status
0.560822 Spectre V2 : Mitigation: Enhanced / Automatic IBRS
0.562821 RETBleed: Mitigation: Enhanced IBRS
0.564821 ITS: Mitigation: Aligned branch/return thunks
0.565821 MMIO Stale Data: Mitigation: Clear CPU buffers
0.566821 Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
0.568830 Spectre V2 : Spectre v2 / PBRSB-eIBRS: Retire a single CALL on VMEXIT
0.569826 Spectre V2 : Enabling IBPB for BPF
0.571821 Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
0.572825 active return thunk: its_return_thunk
0.574821 Spectre V2 : Spectre BHI mitigation: SW BHB clearing on syscall and VM exit
0.576864 x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
0.578822 x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
0.579822 x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
0.581821 x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
0.582821 x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
0.608971 Freeing SMP alternatives memory: 44K
0.609823 pid_max: default: 32768 minimum: 301
0.610884 LSM: initializing lsm=capability,landlock,yama,bpf,ima
0.611902 landlock: Up and running.
0.613821 Yama: becoming mindful.
0.616928 LSM support for eBPF active
0.617956 Mount-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
0.619861 Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
0.623358 smpboot: CPU0: Intel(R) Core(TM) i9-10900K CPU @ 3.70GHz (family: 0x6, model: 0xa5, stepping: 0x5)
0.623819 Performance Events: unsupported CPU family 6 model 165 no PMU driver, software events only.
0.623819 signal: max sigframe size: 1776
0.623819 rcu: Hierarchical SRCU implementation.
0.623819 rcu:   Max phase no-delay instances is 400.
0.623819 NMI watchdog: Perf NMI watchdog permanently disabled
0.623819 smp: Bringing up secondary CPUs ...
0.623819 smp: Brought up 1 node, 1 CPU
0.623819 smpboot: Total of 1 processors activated (7400.00 BogoMIPS)
0.623819 Memory: 684072K/785868K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 93464K reserved, 0K cma-reserved)
0.623819 devtmpfs: initialized
0.623819 x86/mm: Memory block size: 128MB
0.623819 posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)
0.623819 futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).
0.623819 pinctrl core: initialized pinctrl subsystem
0.623819 PM: RTC time: 18:27:32, date: 2026-09-29
0.623819 NET: Registered PF_NETLINK/PF_ROUTE protocol family
0.623819 DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations
0.623819 DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
0.623819 DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
0.623819 audit: initializing netlink subsys (disabled)
0.623819 thermal_sys: Registered thermal governor 'fair_share'
0.623819 thermal_sys: Registered thermal governor 'bang_bang'
0.623819 thermal_sys: Registered thermal governor 'step_wise'
0.623819 thermal_sys: Registered thermal governor 'user_space'
0.623819 thermal_sys: Registered thermal governor 'power_allocator'
0.623819 audit: type=2000 audit(1790711165.382:1): state=initialized audit_enabled=0 res=1
0.623819 cpuidle: using governor menu
0.623819 acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
0.623819 PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]
0.623819 PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry
0.623819 PCI: Using configuration type 1 for base access
0.623819 kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
0.623819 HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
0.623819 HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
0.623819 HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
0.623819 HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
0.623819 ACPI: Added _OSI(Module Device)
0.623819 ACPI: Added _OSI(Processor Device)
0.623819 ACPI: Added _OSI(Processor Aggregator Device)
0.623819 ACPI: 1 ACPI AML tables successfully acquired and loaded
0.623819 ACPI: Interpreter enabled
0.623819 ACPI: PM: (supports S0 S3 S4 S5)
0.623819 ACPI: Using IOAPIC for interrupt routing
0.623819 PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
0.623819 PCI: Using E820 reservations for host bridge windows
0.623819 ACPI: Enabled 2 GPEs in block 00 to 3F
0.623819 ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
0.623819 acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
0.623819 acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]
0.623819 acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]
0.623819 PCI host bridge to bus 0000:00
0.623819 pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
0.623819 pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
0.623819 pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
0.623819 pci_bus 0000:00: root bus resource [mem 0x30000000-0xafffffff window]
0.623819 pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
0.623819 pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]
0.623819 pci_bus 0000:00: root bus resource [bus 00-ff]
0.623819 pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint
0.623819 pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint
0.623819 pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]
0.623819 pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]
0.623819 pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]
0.623819 pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
0.623819 pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint
0.623819 pci 0000:00:02.0: BAR 0 [io  0xc180-0xc19f]
0.623819 pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]
0.623819 pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]
0.623819 pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]
0.623819 pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint
0.623819 pci 0000:00:03.0: BAR 0 [io  0xc1a0-0xc1bf]
0.623819 pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]
0.623819 pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]
0.623819 pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint
0.623819 pci 0000:00:04.0: BAR 0 [io  0xc000-0xc07f]
0.623819 pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]
0.623819 pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]
0.623819 pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint
0.623819 pci 0000:00:05.0: BAR 0 [io  0xc080-0xc0ff]
0.623819 pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]
0.623819 pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]
0.623819 pci 0000:00:06.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint
0.623819 pci 0000:00:06.0: BAR 0 [io  0xc1c0-0xc1df]
0.623819 pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]
0.623819 pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]
0.623819 pci 0000:00:06.0: ROM [mem 0xfeb80000-0xfebbffff pref]
0.623819 pci 0000:00:07.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint
0.623819 pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]
0.623819 pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]
0.623819 pci 0000:00:08.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint
0.623819 pci 0000:00:08.0: BAR 0 [io  0xc100-0xc13f]
0.623819 pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]
0.623819 pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]
0.623819 pci 0000:00:09.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint
0.623819 pci 0000:00:09.0: BAR 0 [io  0xc1e0-0xc1ff]
0.623819 pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]
0.623819 pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]
0.623819 pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint
0.623819 pci 0000:00:1d.0: BAR 4 [io  0xc200-0xc21f]
0.623819 pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint
0.623819 pci 0000:00:1d.1: BAR 4 [io  0xc220-0xc23f]
0.623819 pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint
0.623819 pci 0000:00:1d.2: BAR 4 [io  0xc240-0xc25f]
0.623819 pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint
0.623819 pci 0000:00:1d.7: BAR 0 [mem 0xfebd9000-0xfebd9fff]
0.623819 pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint
0.623819 pci 0000:00:1f.0: quirk: [io  0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO
0.623819 pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint
0.623819 pci 0000:00:1f.2: BAR 4 [io  0xc260-0xc27f]
0.623819 pci 0000:00:1f.2: BAR 5 [mem 0xfebda000-0xfebdafff]
0.623819 pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint
0.623819 pci 0000:00:1f.3: BAR 4 [io  0x0700-0x073f]
0.623819 ACPI: PCI: Interrupt link LNKA configured for IRQ 10
0.623819 ACPI: PCI: Interrupt link LNKB configured for IRQ 10
0.623819 ACPI: PCI: Interrupt link LNKC configured for IRQ 11
0.623819 ACPI: PCI: Interrupt link LNKD configured for IRQ 11
0.623819 ACPI: PCI: Interrupt link LNKE configured for IRQ 10
0.623819 ACPI: PCI: Interrupt link LNKF configured for IRQ 10
0.623819 ACPI: PCI: Interrupt link LNKG configured for IRQ 11
0.623819 ACPI: PCI: Interrupt link LNKH configured for IRQ 11
0.623819 ACPI: PCI: Interrupt link GSIA configured for IRQ 16
0.623819 ACPI: PCI: Interrupt link GSIB configured for IRQ 17
0.623819 ACPI: PCI: Interrupt link GSIC configured for IRQ 18
0.623819 ACPI: PCI: Interrupt link GSID configured for IRQ 19
0.623819 ACPI: PCI: Interrupt link GSIE configured for IRQ 20
0.623819 ACPI: PCI: Interrupt link GSIF configured for IRQ 21
0.623819 ACPI: PCI: Interrupt link GSIG configured for IRQ 22
0.623819 ACPI: PCI: Interrupt link GSIH configured for IRQ 23
0.623819 iommu: Default domain type: Translated
0.623819 iommu: DMA domain TLB invalidation policy: lazy mode
0.623819 ACPI: bus type USB registered
0.623819 usbcore: registered new interface driver usbfs
0.623819 usbcore: registered new interface driver hub
0.623819 usbcore: registered new device driver usb
0.623819 NetLabel: Initializing
0.623819 NetLabel:  domain hash size = 128
0.623819 NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
0.623819 NetLabel:  unlabeled traffic allowed by default
0.623819 PCI: Using ACPI for IRQ routing
0.623819 pci 0000:00:01.0: vgaarb: setting as boot VGA device
0.623819 pci 0000:00:01.0: vgaarb: bridge control possible
0.623819 pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
0.623819 vgaarb: loaded
0.623819 hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
0.623819 hpet0: 3 comparators, 64-bit 100.000000 MHz counter
0.623819 clocksource: Switched to clocksource kvm-clock
0.623819 VFS: Disk quotas dquot_6.6.0
0.623819 VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
0.623819 pnp: PnP ACPI init
0.623819 ACPI: IRQ 4 override to edge(!), high(!)
0.623819 system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved
0.623819 pnp: PnP ACPI: found 5 devices
0.623819 clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
0.623819 clocksource: Switched to clocksource acpi_pm
0.623819 NET: Registered PF_INET protocol family
0.623819 IP idents hash table entries: 16384 (order: 5, 131072 bytes, linear)
0.623819 tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes, linear)
0.623819 Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
0.623819 TCP established hash table entries: 8192 (order: 4, 65536 bytes, linear)
0.623819 TCP bind hash table entries: 8192 (order: 6, 262144 bytes, linear)
0.623819 TCP: Hash tables configured (established 8192 bind 8192)
0.623819 MPTCP token hash table entries: 1024 (order: 3, 24576 bytes, linear)
0.623819 UDP hash table entries: 512 (order: 3, 32768 bytes, linear)
0.623819 UDP-Lite hash table entries: 512 (order: 3, 32768 bytes, linear)
0.623819 NET: Registered PF_UNIX/PF_LOCAL protocol family
0.623819 NET: Registered PF_XDP protocol family
0.623819 pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
0.623819 pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
0.623819 pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
0.623819 pci_bus 0000:00: resource 7 [mem 0x30000000-0xafffffff window]
0.623819 pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]
0.623819 pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]
0.623819 ACPI: \_SB_.GSIA: Enabled at IRQ 16
0.623819 ACPI: \_SB_.GSIB: Enabled at IRQ 17
0.623819 ACPI: \_SB_.GSIC: Enabled at IRQ 18
0.623819 ACPI: \_SB_.GSID: Enabled at IRQ 19
0.623819 PCI: CLS 0 bytes, default 64
0.623819 clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x6aaaabc71c7, max_idle_ns: 881590412124 ns
0.623819 Trying to unpack rootfs image as initramfs...
0.623819 Freeing initrd memory: 29052K
0.623819 Initialise system trusted keyrings
0.623819 workingset: timestamp_bits=40 max_order=18 bucket_order=0
0.623819 Key type asymmetric registered
0.623819 Asymmetric key parser 'x509' registered
0.623819 Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)
0.623819 io scheduler mq-deadline registered
0.623819 io scheduler kyber registered
0.623819 Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
0.623819 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
0.623819 Linux agpgart interface v0.103
0.623819 ACPI: bus type drm_connector registered
0.623819 usbcore: registered new interface driver usbserial_generic
0.623819 usbserial: USB Serial support registered for generic
0.623819 intel_pstate: CPU model not supported
0.623819 drop_monitor: Initializing network drop monitor service
0.623819 NET: Registered PF_INET6 protocol family
0.623819 Segment Routing with IPv6
0.623819 In-situ OAM (IOAM) with IPv6
0.623819 IPI shorthand broadcast: enabled
0.623819 sched_clock: Marking stable (146308617, 476511008)->(687065160, -64245535)
1.972780 registered taskstats version 1
1.975424 Loading compiled-in X.509 certificates
1.980869 Demotion targets for Node 0: null
1.983354 Key type .fscrypt registered
1.985561 Key type fscrypt-provisioning registered
1.988296 ima: No TPM chip found, activating TPM-bypass!
1.991297 ima: Allocated hash algorithm: sha1
1.993878 ima: No architecture policies found
1.996384 PM:   Magic number: 10:36:495
1.999412 RAS: Correctable Errors collector initialized.
2.005380 clk: Disabling unused clocks
2.007670 PM: genpd: Disabling unused power domains
2.014393 Freeing unused decrypted memory: 2028K
2.021328 Freeing unused kernel image (initmem) memory: 3652K
2.024929 Write protecting the kernel read-only data: 32768k
2.029995 Freeing unused kernel image (text/rodata gap) memory: 1184K
2.034612 Freeing unused kernel image (rodata/data gap) memory: 720K
2.045645 x86/mm: Checked W+X mappings: passed, no W+X pages found.
2.049326 Run /init as init process
2.058363 systemd[1]: Inserted module 'autofs4'
2.077181 fuse: init (API version 7.45)
2.084097 ACPI: \_SB_.GSIG: Enabled at IRQ 22
2.088862 ACPI: \_SB_.GSIH: Enabled at IRQ 23
2.094911 ACPI: \_SB_.GSIE: Enabled at IRQ 20
2.099640 ACPI: \_SB_.GSIF: Enabled at IRQ 21
error: interrupted by the user

A bit more context about this situation;

  • Hyper-V exposes virtualization to guest (L1)
    Get-VMProcessor “llm-host” | fl ExposeVirtualizationExtensions

    ExposeVirtualizationExtensions : True

  • lscpu shows hypervisor flag

  • kvm_intel module has clamped value N for enable_apicv

(L1) $ lscpu
Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             39 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      10
  On-line CPU(s) list:       0-9
Vendor ID:                   GenuineIntel
  Model name:                Intel(R) Core(TM) i9-10900K CPU @ 3.70GHz
    CPU family:              6
    Model:                   165
    Flags:                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr ss
                             e sse2 ss ht syscall nx pdpe1gb rdtscp lm rep_good nopl xtopology cpuid aperfmperf tsc_know
                             n_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave av
                             x f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced tpr_s
                             hadow ept vpid ept_ad fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt
                             xsaveopt xsavec xgetbv1 xsaves vnmi md_clear flush_l1d arch_capabilities
Virtualization features:
  Virtualization:            VT-x
  Hypervisor vendor:         Microsoft
  Virtualization type:       full
(L1) $ head -n -0 /sys/module/kvm_intel/parameters/\\\*

Trimmed output

==> /sys/module/kvm_intel/parameters/emulate_invalid_guest_state <==
Y
==> /sys/module/kvm_intel/parameters/enable_apicv <==
N
==> /sys/module/kvm_intel/parameters/enable_device_posted_irqs <==
N
==> /sys/module/kvm_intel/parameters/enable_ipiv <==
N
==> /sys/module/kvm_intel/parameters/enable_shadow_vmcs <==
N
==> /sys/module/kvm_intel/parameters/enlightened_vmcs <==
Y
==> /sys/module/kvm_intel/parameters/ept <==
Y
==> /sys/module/kvm_intel/parameters/eptad <==
Y
==> /sys/module/kvm_intel/parameters/error_on_inconsistent_vmcs_config <==
Y
==> /sys/module/kvm_intel/parameters/fasteoi <==
Y
==> /sys/module/kvm_intel/parameters/flexpriority <==
N
==> /sys/module/kvm_intel/parameters/nested <==
Y
==> /sys/module/kvm_intel/parameters/preemption_timer <==
N
==> /sys/module/kvm_intel/parameters/sgx <==
N
==> /sys/module/kvm_intel/parameters/unrestricted_guest <==
Y
==> /sys/module/kvm_intel/parameters/vmentry_l1d_flush <==
not required
==> /sys/module/kvm_intel/parameters/vnmi <==
Y
==> /sys/module/kvm_intel/parameters/vpid <==
Y

L2 config is the straight forward empty case;

{ lib, hostPkgs }:
lib.nixos.runTest {
  inherit hostPkgs;
  name = "platform-memory-posture";

  nodes.machine =
    { lib, ... }:
    {
      imports = [ ];
      virtualisation.memorySize = 768;
      # No virtiofs mounts
      virtualisation.useNixStoreImage = true;
      virtualisation.sharedDirectories = lib.mkForce { };
      virtualisation.qemu.options = [ ];
    };

  testScript = ''
    machine.start()
    machine.wait_for_unit("multi-user.target")
  '';
}