Hey there, I’m in the process of moving my home lab from multiple ubuntu machines running services in containers all managed with ansible, to NixOS machines. Basic setup is great so far, but I’m still at the ‘managing multiple configuration.nix files via git pulls’ stage, and I’d like to remotely declare the state of the cluster and run a command to build / switch to the new intended state.
What is the state of the art therefore of remote declarative management? I had a look at NixOps but the documentation isn’t great and it looks a little ‘old’? I stumbled across Morph and that seems to do something similar. Finally there is still ansible, as I can still run remote git pull and ‘nixos-rebuild switch’, but ansible is slow AF and I haven’t actually managed to make it work on NixOS target hosts yet as there is some python package issue…
For a homelab style thing I’d personally not actually use anything nearly as complicated as ansible or nixops though. Are you aware of nix-copy-closure and nixos-rebuild’s --target-host switch?
A simple flake with multiple configurations, and deploying them all with nixos-rebuild switch --target-host would be doable. Tools more like deploy-rs and colmena would probably be more than enough to make the workflow convenient.
Unless you have hundreds of machines to deploy, or security requirements that make you not want to collect all ssh keys on a single host, a push-style workflow seems perfectly appropriate.
Then just run all that through CI jobs for a full gitops workflow.
I’m trying to come up with a setup that helps me automate maintaining notebooks and one or two servers in a smaller office environment. It’s a setup that I’ve traditionally done with Puppet (or Ansible) and The Foreman as an external node classifier (ENC).
The Foreman is a user-friendly front-end for monitoring and managing the host automation. Apart from showing host statistics (e.g. collected by Puppet) it makes it possible to mark a host for re-installation, and upon reboot that machine receives a boot image via PXE and TFTP (or similar) to trigger a fully-automatic installation of the host.
I’m wondering if anyone has ever done something like that before with Nix. In the Foreman community NixOS doesn’t seem to be on anyone’s radar.
NixOS effectively reinstalls itself on every boot, so assuming a reasonable configuration you can just wipe the (non nix-store, to get rid of data) partitions on it to achieve the same effect. This is a pretty common setup, popularized by@grahamc
There’s no UI or other architecture to do this across a fleet to my knowledge, but some kind of NixOS plugin for a fleet-management engine would probably be pretty easy to implement. The main difficulty will likely be making the fleet-management thing correctly deal with a non-FHS system, which they’re pretty bad at in general. NixOS does also significantly reduce the value of their features though, since a lot of them become plain unnecessary.
Can you please just clear one (probably obvious) point for me: before i can use nixos-rebuild --target-host, or colmena, or deploy-rs - I do need to manually install nixos on every machine, with a user with main host’s public ssh key (in authorized_keys), right?
…sorry, and yes you’d need ssh in authorized keys on each machine too (so you could try to do that in the initial configuration/installation)
Sometimes people do these first setup/provision with a companion tool like terraform or pulumi.
In other cases, people may have the ability to create an image for the hosting provider ahead of time (often this can be created with nix too from a flake nixosConfiguration)
