NixOS cloud images available?

Help
1

Hi all!

I’ve been using NixOS on the desktop for a few months and want to use it more for servers as well. I wondered if there are original cloud images floating around? I’d like to add them to my Proxmox cluster as a template.

Currently, my workflow for deploying a Debian or Ubuntu machine is this:

  • Clone the Debian official cloud image;
  • Set cloudinit details;
  • Make disk larger;
  • Start the VM.

For NixOS, I currently don’t have a workflow like that.

Instead, I’ve done one or two VMs with nixos-generate and the minimal ISO, but it isn’t ideal. The thing is I’d like to offer the template to multiple users and things like username, ssh key and network configuration are fluid and can change with every deployment. Simply building an image with nixos-generate with hard-coded settings doesn’t cut it.

How are other people doing it? Am I missing something obvious?

2

You don’t have to hardcode the settings. We have a pve module that will let you create VMA files that can be directly restored as preconfigured VMs using qmrestore

Here’s a sample flake that shows the whole process of building and importing these cloud images:

{
        description = "A very basic flake";

        inputs.nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";

        outputs = { self, nixpkgs }: let
                pve_ssh_tgt = "root@192.168.1.150";
                vmid = "9918";
                vm_ip = "gw=192.168.1.1,ip=192.168.1.111/24";
        in {
                nixosConfigurations.test-vm = nixpkgs.lib.nixosSystem {
                        system = "x86_64-linux";
                        modules = [
                                ({lib, modulesPath, pkgs, ...}: {
                                        imports = [ "${modulesPath}/virtualisation/proxmox-image.nix" ];
                                        # These are only to populate the cloud-init default config
                                        # It can be changed on the PVE UI
                                        proxmox = {
                                                qemuConf = {
                                                        name = "test-vm";
                                                        cores = 2;
                                                        memory = 2048;
                                                };
                                                qemuExtraConf.ipconfig0 = vm_ip;
                                        };
                                        networking = {
                                                hostName = lib.mkForce "";
                                                useDHCP = false;
                                        };
                                        services = {
                                                cloud-init = {
                                                        enable = true;
                                                        network.enable = true;
                                                };
                                                getty.autologinUser = "root";
                                                sshd.enable = true;
                                        };
                                })
                        ];
                };
                packages.x86_64-linux.deploy = nixpkgs.legacyPackages.x86_64-linux.writeScriptBin "deploy" ''
                        VMA=$(tail -n1 ${
                                self.nixosConfigurations.test-vm.config.system.build.VMA
                        }/nix-support/hydra-build-products | awk '{print $NF}')
                        [ -z $VMA ] && false
                        scp $VMA ${pve_ssh_tgt}:/var/lib/vz/dump/vzdump-qemu-nixos-test.vma.zst
                        ssh ${pve_ssh_tgt} "qm stop ${vmid} && qm destroy ${vmid}; qmrestore local:backup/vzdump-qemu-nixos-test.vma.zst 9918 --unique --storage local-lvm && qm set ${vmid} --ide2 local-lvm:cloudinit && qm start ${vmid}"
                '';

        };
}

Update the parameters in the let block and run nix run .#deploy to test it out.

If you don’t want to have to build the VMA yourself, hydra builds them too. However, they don’t have cloud-init enabled, and are kinda useless out of the box. I’ll open a PR to enable cloud init by default.

EDIT: opened the PR

2 Likes
3

Thank you for this!

I noticed you specified some VM specific settings in your config. Mainly a name and core count.

The goal is to create a Proxmox template that me and other users can clone, and get access to via the settings issued through cloud-init. This makes it very fast for everyone to spin up new NixOS VMs and is more akin to our current workflow for Debian and Ubuntu images.

4

Btw, this works vv well for me. it converts VMs into running NixOS. A really neat little tool.

5

Hi everyone, what is the default username and password for Hydra build?

6

Bumping this thread again.

Problem is that I don’t want to have to prepare images beforehand. They should come from NixOS themselves. I think this can really help with the adoption.

Ubuntu, Debian, Fedora and other distributions already offer ready-to-go Cloud Images.

What I’d like to do (and already do with Debian) is akin to this: Cloud-Init Support - Proxmox VE

7

Sorry that took so long; I finally got around to this. WIP PR: nixos/proxmox-image: add cloudImage build output by illustris · Pull Request #429178 · NixOS/nixpkgs · GitHub

I’ll test it a bit more before marking it as ready. This should make PVE compatible cloud images available on hydra.

No default creds are set. You can set them using cloud-init.

8

This looks great but how is it supposed to be used? I didn’t find any updated docs in the PR (worth to mention that I’m new to NixOS)

9

Some day, long after we’ve all turned to dust and returned to the earth, I hope this PR will get merged. On that day, our descendants can go to hydra.nixos.org to download the image named nixos.proxmoxCloudImage.x86_64-linux. They can also run nix build .#nixosConfogurations.<machine name>.config.system.build.proxmoxCloudImage. Meanwhile we will have to explicitly import the version of the module from my PR to be able to build this output.

10

I wish the image would not be Proxmox specific and that the first configuration.nix gets made by the cloud-init process.

I also wish NixOS itself would supply these images just like other major Linux distributions do. Like Ubuntu, Debian, Fedora.

The absence of cloud-init images is why I am not currently offering NixOS as an option in my VPS portal. I just don’t have the time to research and maintain the current way Nix does cloud images. I really hope that will change someday.