In order to see if a bug is only on a raspberry pi or on any aarch64 device I wanted to run a VM within NixOs. To do that I enabled boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; and I run:
$ nixos-rebuild build-vm --flake 'git+https://gist.github.com/tobiasBora/8fa251d169af567fe258cffe9e048054?ref=main#lxqtSdcard'
$ ./result/bin/run-nixos-vm
kvm version too old
qemu-system-aarch64: failed to initialize kvm: Function not implemented
qemu-system-aarch64: falling back to tcg
/nix/store/396yl0rc18faf30nc4ar58m3xam02r0v-gvfs-1.50.0/lib/gio/modules/libgvfsdbus.so: cannot open shared object file: No such file or directory
Failed to load module: /nix/store/396yl0rc18faf30nc4ar58m3xam02r0v-gvfs-1.50.0/lib/gio/modules/libgvfsdbus.so
Gtk-Message: 09:52:09.821: Failed to load module "colorreload-gtk-module"
Gtk-Message: 09:52:09.826: Failed to load module "window-decorations-gtk-module"
Then a qemu window starts (and my fans start, so the CPU is working), but it prints Guest has not initialized the display (yet) even after a few minutes. Any idea what I did wrong? Or is it just that qemu is to slow to run a whole aarch64 vm on x86_64?
I think it’s just very (very) slow. You can pass -serial stdio to run-nixos-vm to see the boot logs over serial. I ran it for about half an hour with -smp 16 -m 4096, and it is still in the boot process.
kvm version too old
qemu-system-aarch64: failed to initialize kvm: Function not implemented
qemu-system-aarch64: falling back to tcg
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x000f0510]
[ 0.000000] Linux version 6.0.7 (nixbld@localhost) (gcc (GCC) 9.5.0, GNU ld (GNU Binutils) 2.39) #1-NixOS SMP Thu Nov 3 15:00:35 UTC 2022
[ 0.000000] random: crng init done
[ 0.000000] Machine model: linux,dummy-virt
[ 0.000000] efi: UEFI not found.
[ 0.000000] NUMA: No NUMA configuration found
[ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000013fffffff]
[ 0.000000] NUMA: NODE_DATA [mem 0x13f7ef080-0x13f7f1fff]
.
.
.
[ 1644.855119] 8021q: 802.1Q VLAN Support v1.8
[ 1691.174268] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[ 1730.654097] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
2 Likes
Oh thanks a lot! Is there any reasons why this is so slow considering the fact that binfmt is actually way faster (x5 maybe) than my raspberry pi when I switch to a new system?
I believe that you’re building a script that runs the aarch64 build of qemu, so your system is happily emulating qemu itself using qemu. Double emulation. build-vm will build a script native to the platform of the system you’re building; i.e. aarch64 in this case.
2 Likes
Oh make sense good point… So how could I use the x86 version of qemu to run the aarch64 system?
EDIT: I tried to check and indeed it’s exactly what you describe:
$ file /nix/store/r4glhlf5pd4avbw9r0mwhsixjfzvvsqj-qemu-host-cpu-only-7.1.0/bin/qemu-system-aarch64
/nix/store/r4glhlf5pd4avbw9r0mwhsixjfzvvsqj-qemu-host-cpu-only-7.1.0/bin/qemu-system-aarch64: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/kjnfc4rah46ilnk7iii6g6zwbmphx0vr-glibc-2.35-163/lib/ld-linux-aarch64.so.1, for GNU/Linux 2.6.32, not stripped
I will try to manually edit the nix-run-script to see how it goes.
So indeed the problem is the one given by ElvishJerricco: there are two layers of emulation. So copy/pasting the nix-run-vm + edit in the script so that all paths points to their x86 version + using -serial stdio -smp 16 -m 4096 as recommended by illustris, I was able to boot the whole GUI in 8 minutes (still slower than a raspberry pi… but way better than before, at least I can start GUI apps in a reasonable time). Now I get other bugs with some GUI tools, but it’s another issue.
Thanks a lot for the help, let me know if you know a better solution that manually editing the file.
1 Like