Thanks for your willingness to help!
I’m starting with just the pre-made aarch64 sd image from hydra.
Ah right I see it now in the first post.
Skipped it a bit too fast :).
I had the same problems as you with your example config.
I gave nixos-rebuild build-vm a try and in that VM they keyboard works fine and I have internet.
I check tomorrow what other options that VM uses.
Huh. So is that from an x86 NixOS in both cases?
Yes just a quick test with the config of my notebook to see if there maybe is a general problem.
The problem for the ethernet might be this one here:
I looked at the issue from 虎游 @hu60.cn about the dwc2 problem, and it looks like the issue there is that he/she is using a modular kernel, but the root disk image does not contain any modules for that kernel. It looks like at least cdc_ether.ko and cdc_subset.ko are needed for the ethernet to work. I checked that by using the 2020-08-20-raspios-buster image that is referenced on the page where they downloaded that kernel, and with that the network comes up.
When I build my Raspi Flake I can see the files in the result.
However I’m not good enough to know if they are in the image provided by NixOS.
As for the keyboard issue I found out that it seems to be working without a problem in the uboot menu but then failes at a later stage.
And with these parameters you don’t have to search for the bus number:
-usb -device usb-mouse -device usb-kbd \
I tried this project and it works just fine (SSH and keyboard) and the qemu command looks very similar to yours.
I’m going to test the parameters to see if the help but doubt it.
My guess is more that we need a different NixOS image for this.
Edit: I replaced the images from the repo with the kernel and image from NixOS and get the same behavior.
The same working behavior (as with the repo’s original image)? Or the same non-working behavior (as with the NixOS image)?
The same non-working behavior.
Right, I seem to be having a hard time reading 
Well at least we got the same behavior 
I’m currently trying to build my config as an image and see if that boots.
1 Like
Well and that didn’t work either.
I reckon I’m now as lost as you are.
Maybe @samueldr knows if the problem from the qemu Gitlab repository is related?
nix build .#images.raspi
error: builder for '/nix/store/9rm1hqc9d3paphpx8pmfzw3i1bc5wla2-linux-5.15.32-1.20220331-modules-shrunk.drv' failed with exit code 1;
last 10 log lines:
> copying dependency: /nix/store/zmpsqch7hz0zkjgg1fjcpr3f7vf1dzi3-linux-5.15.32-1.20220331-modules/lib/modules/5.15.32/kernel/drivers/message/fusion/mptscsih.ko.xz
> copying dependency: /nix/store/zmpsqch7hz0zkjgg1fjcpr3f7vf1dzi3-linux-5.15.32-1.20220331-modules/lib/modules/5.15.32/kernel/drivers/message/fusion/mptspi.ko.xz
> root module: vmxnet3
> copying dependency: /nix/store/zmpsqch7hz0zkjgg1fjcpr3f7vf1dzi3-linux-5.15.32-1.20220331-modules/lib/modules/5.15.32/kernel/drivers/net/vmxnet3/vmxnet3.ko.xz
> root module: vsock
> copying dependency: /nix/store/zmpsqch7hz0zkjgg1fjcpr3f7vf1dzi3-linux-5.15.32-1.20220331-modules/lib/modules/5.15.32/kernel/net/vmw_vsock/vsock.ko.xz
> root module: simplefb
> builtin dependency: simplefb
> root module: sun4i-drm
> modprobe: FATAL: Module sun4i-drm not found in directory /nix/store/zmpsqch7hz0zkjgg1fjcpr3f7vf1dzi3-linux-5.15.32-1.20220331-modules/lib/modules/5.15.32
For full logs, run 'nix log /nix/store/9rm1hqc9d3paphpx8pmfzw3i1bc5wla2-linux-5.15.32-1.20220331-modules-shrunk.drv'.
error: 1 dependencies of derivation '/nix/store/qfwf3zkcf5h791gab0lnwzzybgyk8xi5-stage-1-init.sh.drv' failed to build
error: 1 dependencies of derivation '/nix/store/nzqsmvp7gvim681bpxvv3y485gkk9jq1-initrd-linux-5.15.32-1.20220331.drv' failed to build
error: 1 dependencies of derivation '/nix/store/8ywv7haf8b10kys3vx1zkah4gci638m6-nixos-system-raspi-test-22.05.20220623.ccf8bdf.drv' failed to build
error: 1 dependencies of derivation '/nix/store/66irxbxlcgylja92c1ih6dpi1kkj6n96-ext4-fs.img.zst.drv' failed to build
error: 1 dependencies of derivation '/nix/store/wdkjfpf7mbhvldgz0wzs0jk8iqs22sz9-nixos-sd-image-22.05.20220623.ccf8bdf-aarch64-linux.img.drv' failed to build
@lheckemann has some awesome gists: lheckemann’s gists · GitHub
For example, an aarch64 VM that builds and runs from my x86 host, with working keyboard!: nix-build aarch64-vm.nix https://gist.github.com/lheckemann/63c52f2115346e6c9bbc6ecdfde9f43b#file-aarch64-vm-nix
It’s not using the raspi3b machine in qemu though.
Also, one that will probably be helpful for my BTRFS Pi (mis)adventures: https://gist.github.com/lheckemann/f265f155e9e7a7d05028eacfa6e96114
2 Likes
With these (minor) changes, I have a booting aarch64 image that I can ssh into (at 2222). Hooray!
{ pkgsPath ? <nixpkgs> }:
let
pkgs = import pkgsPath {};
pkgsAarch64 = import pkgsPath { system = "aarch64-linux"; };
iso = (pkgsAarch64.nixos {
imports = [ (pkgsPath + "/nixos/modules/installer/cd-dvd/installation-cd-base.nix") ];
users.users.root.openssh.authorizedKeys.keyFiles = [(builtins.fetchurl https://github.com/n8henrie.keys)];
}).config.system.build.isoImage;
vmScript = pkgs.writeScript "run-nixos-vm" ''
#!${pkgs.runtimeShell}
${pkgs.qemu}/bin/qemu-system-aarch64 \
-machine virt,gic-version=max \
-cpu max \
-m 2G \
-smp 4 \
-drive file=$(echo ${iso}/iso/*.iso),format=raw,readonly=on \
-nic user,hostfwd=tcp::2222-:22 \
-nographic \
-bios ${pkgsAarch64.OVMF.fd}/FV/QEMU_EFI.fd
'';
in vmScript
A few questions and issues:
- what is this all about?
-bios ${pkgsAarch64.OVMF.fd}/FV/QEMU_EFI.fd - currently I’m using this to run it:
nix-build aarch64-vm.nix && ./result– there’s got to be a better way withnix runornix-shell --command, right? - can I get this working with a raspi3b emulated machine?
- will this work with KVM so it’s hopefully not so slow?
Oh wow, I didn’t even know I’d made that VM gist. Glad it’s proving useful!
OVMF is a UEFI implementation for virtualised environments. You need some sort of “firmware” for the machine, and this is what provides it.
If you replace writeScript with writeScriptBin, the package will produce a directory containing bin/run-nixos-vm instead of placing the script directly in the store. You can use that with nix-shell -p '(import ./aarch64-vm.nix)' --command "run-nixos-vm", or restructure the whole thing into a flake which you can use with nix run, but I don’t think there’s that much benefit in either of those approaches compared to what you’re doing now.
Not on x86, since KVM relies on the CPU being able to execute the machine code directly. On aarch64 hosts, it can work, but I haven’t tried this myself and don’t know what exactly one might have to do to achieve it (might work without any extra steps, might require some extra command line args for qemu).
1 Like
No luck with networking using this as a template and the raspi3b qemu “machine” (had to change the netdev back to usb-net, which is what works on raspios).
I had to manually copy the iso so that I could (mutably) resize it with qemu-img. With that and the usb-net config (that again works for RaspiOS) I get the same error message as above, and no connectivity:
qemu: Slirp: Failed to send packet, ret: -1
Darn.
I don’t know about the Raspi3 but the Raspi4 can boot normal UEFI with the correct bootloader.
However for my use case this wouldn’t be really that useful if it works with QEMU because it was broken on an actual Raspi4. So I would be testing something that I don’t really use.
Well maybe this will help others reproduce the issue at least?
I get the same behavior with this when I attempt to ssh in. I also don’t see any boot stuff in the popup window, interestingly.
It does require some mutability to copy the sd-image to a local file and chmod 0640 – qemu apparently won’t boot an sd image without write permissions.
{ pkgsPath ? <nixpkgs> }:
let
pkgs = import pkgsPath { };
pkgsAarch64 = import pkgsPath { system = "aarch64-linux"; };
sdImage = (pkgsAarch64.nixos {
imports = [
(pkgsPath + "/nixos/modules/installer/sd-card/sd-image-aarch64-installer.nix")
(pkgsPath + "/nixos/modules/profiles/qemu-guest.nix")
];
boot.kernelParams = [ "console=tty1" "console=ttyAMA0" "console=ttyS0,115200" ];
users.users.root.openssh.authorizedKeys.keyFiles = [ (builtins.fetchurl https://github.com/n8henrie.keys) ];
services.qemuGuest.enable = true;
}).config.system.build.sdImage.overrideAttrs (oldAttrs: {
compressImage = false;
});
qemuImage = pkgs.stdenv.mkDerivation
{
name = "aarch64-qemu.img";
dontUnpack = true;
installPhase = ''
img=./nixos-aarch64.img
cp ${sdImage}/sd-image/*.img "$img"
chmod 0640 "$img"
${pkgs.qemu}/bin/qemu-img resize -f raw "$img" 4G
cp "$img" $out
'';
};
vmScript = pkgs.writeScript "run-nixos-vm" ''
#!${pkgs.runtimeShell}
img=aarch64-qmu.img
cp "${qemuImage}" "$img"
chmod 0640 "$img"
${pkgs.qemu}/bin/qemu-system-aarch64 \
-machine raspi3b \
-cpu max \
-m 1G \
-smp 4 \
-drive file="$img",format=raw \
-device usb-net,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::2222-:22 \
-bios ${pkgsAarch64.OVMF.fd}/FV/QEMU_EFI.fd
'';
in
vmScript
No change with the below:
boot.kernelModules = [ "cdc_ether" "cdc_subset" ];
EDIT: or adding usbnet.
Any news regarding this? I would also like to see this issue resolved.