Scripted and unattended libvirt installs with NixOS

I use virt-install to build libvirt domains. I would like to use the “–extra-args” option which requires changing from “–cdrom” to “–location” as described in the man page.

--location allows things like --extra-args for kernel arguments, and using --initrd-inject. If you want to use those options with CDROM media, you can pass the ISO to --location as well which works for some, but not all, CDROM media.

I have noted the last sentence and in my case, swapping;
--cdrom="/srv/iso/NixOS 24.05/nixos-minimal-24.05.1270.2819fffa7fa4-x86_64-linux.iso"
to
--location="/srv/iso/NixOS 24.05/nixos-minimal-24.05.1270.2819fffa7fa4-x86_64-linux.iso"
throws this error, ERROR [Errno 2] No such file or directory: 'xorriso'

Apparently this might be related to being unable to find to the kernel and initrd and there is an option to pass this on the command line like;
--location="/srv/iso/NixOS 24.05/nixos-minimal-24.05.1270.2819fffa7fa4-x86_64-linux.iso,kernel=boot/bzImage,initrd=boot/initrd"

The man page says;
"Additionally, --location can take 'kernel' and 'initrd' sub options. These paths relative to the specified location URL/ISO that allow selecting specific files for kernel/initrd within the install tree. This can be useful if virt-install/ libosinfo doesn't know where to find the kernel in the specified --location."

I presume I have got the paths for the kernel and initrd correct as these came from the ISO but I have tried a variety of others to no avail. This link suggests a method to get the locations from the ISO but it didn’t work for any of the NixOS ISOs I have. It also shows a number of options for other ISOs which seemed quite handy. :slight_smile:

I have also tried using a full install ISO and the ISOs from 23.11 with the same results.

Of course there is always that original note in the man page that it may not work at all. :frowning:

Looking for any advice, experiences or alternatives…

In terms of alternatives, it seems relevant to give the overall background of what I am hoping to achieve,
so my current aim is to do a lot of scripted and unattended libvirt installs (most, but not all are NixOS). For me this means there are four pieces;

  1. Create and partition qcow2 images. I use qemu-img and qemu-nbd which allows me to prep the images before the install in a scripted manner and to upload some files including configuration.nix. I have considered disko and make-disk-image.nix but have only used them to partial success.
  2. Prepare the qemu config which seems to come down to either manually creating the xml file or using virt-install. All the other methods I have looked at seem to prepare disk images that then need to be imported into libvirt/vmm. virt-install works, but if I create a domain with a spice channel then I seem to need to use the virt/remote-viewer window for the install. The option to be able to use a console seems to come down to my earlier question about extra-args.
  3. Perform the actual install. At the moment, this requires a handful of commands. I have looked at uploading some of them as a script when I create the image, but I still need to mount the image. basically they are mount,generate-config (for hardware-configuration.nix), replace configuration.nix with the file I uploaded and do a nixos-install. I have been looking at if the kickstart option in virt-install could do this, but again it requires “–location” and I am not entirely sure if it requires a custom ISO to make it work. I have also been looking at a variation of the method described by Daniel (@573) (Thanks :+1:) in discourse which built a successful image for me. I also looked at how the tutorial on nix.dev built a VM for reference and disko-images seems interesting.
  4. Finally, the key to speed is to use a cache of NixOS packages/derivations as well as other cache servers for the relevant platforms or perhaps a more general proxy-cache. This is a work in progress!

Apologies if this was too verbose.

1 Like