I have my (private) GitHub repo with a flake, configurations, and everything. I’m able to (mostly) zero-touch deploy this configuration to any VM (local VMs using tart, VMs on my network, remove VMs in the cloud), and it’s awesome!
The issue that I have is for after the install. My goal (and maybe it’s the wrong goal) is to be able to install to a VM, and then be able to utilize nixos-rebuild on the VM to apply any configuration changes. nixos-anywhere doesn’t include the configuration repository, so now I’m stuck going down the path of how to get my repository onto the VM.
I’m not using nixos-rebuild from my local machine because it’s a Mac, and just has Nix. I’m not seeing where I’m able to “build” a self-contained ISO because of being on an ARM Mac with an intel Linux target.
From the best that I can tell (and mind you, I’m very new to Nix and NixOS), I can create a package that uses fetchFromGitHub to stick my configuration repository in the Nix store.
I guess my questions are
Is this the best way of doing this?
Am I going about this (my goal) entirely the wrong way?
That’s one way, but it’s a somewhat weird way of doing it, since now you cannot change your configuration and only statically rebuild what you already have deployed.
It’d make more sense to write a systemd unit to download it (with WantedBy=first-boot.target or such, or even on a timer to keep it updated).
Finally, you can just run nixos-rebuild --flake github:your/repo and change the VM config that way. I think you can even set the autoUpgrade timers to do that, then you’d effectively have a gitops setup, just need an access key or SSH key with read permissions on the VM (which can be installed with nixos-anywhere, and/or sops-nix/agenix).
All that said cross compilation is very much possible. I’ve never tried how much effort it’d be to do this, but you should be able to use nixos-rebuild 's --remote-target, or something like deploy-rs.
Now that you put it out there, I think I really have 2 separate issues that I’m trying to solve for. A production deployment, where I have my configuation.nix that I’m happy with, I really like the nixos-rebuild --flake github:your/repo solution of just pointing the configuration directly at GitHub.
The other issue (and I think closer to my original struggles) was how to do “rapid development” and iterate quickly on my configuration.nix. To that regard, I actually just found rsync to be a super simplistic solution to getting everything on my testing VM.
In case any others stumble across this in the future, what I’m running is
rsync --delete --mkpath --compress --recursive . root@$(tart ip nixos):/nix/persist/config && \
ssh root@$(tart ip nixos) "nixos-rebuild switch --flake /nix/persist/config"
If you have ssh access to the root user, why not just use nixos-rebuild switch --target-host and maybe --build-host if you really wanted the remote to build as well?