Self-extracting /nix with NixOS VM image

Help
1

Hi,

I would like to create a (stateless) NixOS VM where /nix is on a separate persistent partition, to cache nix derivations there even when the VM is re-created. The partition would be initially blank, so I would like to essentially copy the /nix of the VM image to that partition if needed, before actually booting the system derivation.

I kind of have an idea how I could implement that: run some service (probably during initrd) which runs nix copy /mnt/vmimageroot/nix/store/$systemderivation /mnt/nixpartition/store/$systemderivation, then continue with booting with the partition mounted to /nix.

But I’m wondering if there is already something which does this, or if someone has already tried something similar?

This also seems like a great use case of the local overlay store feature, has anyone tried that for NixOS?

Thanks!

2

Does the store have to be absolutely blank on the machine start? Or do you just want reproducible VMs with as little state as possible? If it’s the latter, you can use impermanence+microvm.nix and mount hosts’s nix store into the VM as described in the microvm docs.

3

I’d like the VM image to be self-contained, so that you can boot it with an (initially empty) attached disk/partition for /nix and it takes care of the rest. The host does not run NixOS, otherwise for sure microvm.nix would make sense!

4

Isn’t this what the (installer) iso does with the nix store squashfs?

5

That’s a good point. It looks like it uses an overlay filesystem: nixpkgs/nixos/modules/installer/cd-dvd/iso-image.nix at d2141817e8e9083ad4338f9a8020cadfb9729648 · NixOS/nixpkgs · GitHub

And then re-creates the db: nixpkgs/nixos/modules/installer/cd-dvd/iso-image.nix at d2141817e8e9083ad4338f9a8020cadfb9729648 · NixOS/nixpkgs · GitHub

It’s not 100% what I have in mind, but it’s for sure a good starting point, thanks!

1 Like