How to leverage build-vm and virt-manager for an easy multi VM setup

leoperegrino · March 7, 2025, 11:13pm

I’m using the nixos-rebuild build-vm command to create virtual machines based on nixosSystem s. It works great and I’m able to edit everything in a single file, issue a single command and a fully configured system appears. This approach, while good for a single VM, lacks a good interface to manage multiple machines and relies too much on the terminal.

Recently, I started messing with virt-manager and would like to, instead of building a single machine with build-vm, build multiple and manage them through virt-manager, where I could start machines in the background, stop, delete and easily configure them after they are initially generated. I don’t want to create an ISO, install and configure each VM.

I know that the build-vm command spits out a result/bin/run-nixos-vm which contains a qemu-kmv command with all the necessary parameters, but I’m not very skilled with VMs so I don’t know how to use or change the virtualisation.* configs to easily use the fully configured VM in virt-manager.

Does anyone have a specific ergonomic approach for this?

waffle8946 · March 7, 2025, 11:29pm

I’m not familiar with qemu, but hopefully this helps you get started: config.system.build.vm (or if you want a bootloader, config.system.build.vmWithBootloader) will contain the realised config in vm form.

github.com

NixOS/nixpkgs/blob/e2f6e5d7bce6aace59acae5543abc8a41fbffeca/pkgs/os-specific/linux/nixos-rebuild/nixos-rebuild.sh#L815-L830


      
          elif [ "$action" = build-vm ]; then
              if [[ -z $buildingAttribute ]]; then
                  pathToConfig="$(nixBuild $buildFile -A "${attr:+$attr.}config.system.build.vm" "${extraBuildFlags[@]}")"
              elif [[ -z $flake ]]; then
                  pathToConfig="$(nixBuild '<nixpkgs/nixos>' -A vm -k "${extraBuildFlags[@]}")"
              else
                  pathToConfig="$(nixFlakeBuild "$flake#$flakeAttr.config.system.build.vm" "${extraBuildFlags[@]}" "${lockFlags[@]}")"
              fi
          elif [ "$action" = build-vm-with-bootloader ]; then
              if [[ -z $buildingAttribute ]]; then
                  pathToConfig="$(nixBuild $buildFile -A "${attr:+$attr.}config.system.build.vmWithBootLoader" "${extraBuildFlags[@]}")"
              elif [[ -z $flake ]]; then
                  pathToConfig="$(nixBuild '<nixpkgs/nixos>' -A vmWithBootLoader -k "${extraBuildFlags[@]}")"
              else
                  pathToConfig="$(nixFlakeBuild "$flake#$flakeAttr.config.system.build.vmWithBootLoader" "${extraBuildFlags[@]}" "${lockFlags[@]}")"
              fi

leoperegrino · March 8, 2025, 5:05pm

Thank you for the help though I think this is not exactly what I needed. I searched a bit more and found a similar question to mine which had a very useful reply. NixVirt appears to be the optimal solution when you need to manually write XML configurations. While this approach works well, I need to read a bit more to see if configurations can be defined externally to my system flake configuration. i.e: when using build-vm, I want to use a separate flake file to define an entirely new NixOS VM system.

I also found information on the nixos-build-vms command, which can create several vms and connect them with just one declaration [1, 2]. This command is nice but still is not managed by a interface such as virt-manager.

Since the parameters of the result of build-vm are declared here, one could create a nix derivation to mimic this shell but generating a XML for virsh, but that’s a whole new challenge.

In the meantime, I think I’ll use the nixos-build-vms command for easily creating a network of VMs until I find a better approach with virt-manager and maybe NixVirt. If find one, I’ll comeback here.