One could probably make it nicer and get the list of hosts directly from the flake but for the moment it is good enough and allows me to focus on the servers themselves.
One thing which I still have to fix is limit password less sudo to the nix command but that isn’t something which is part of the script but part of the NixOS config.
FYI, nixos-rebuild supports remote builds via --target-host. You can run that from your repo and point nixos-config’s NIX_PATH at some machine’s config; i.e. -I nixos-config=configs/machine/default.nix.
I’m not familiar with Cachix, but I think this is a bit too complicated for my use case. I wasn’t looking for a CI system but rather a script to way to deploy on remote machines. Thank you for the offer though!
One could probably make it nicer and get the list of hosts directly from the flake but for the moment it is good enough and allows me to focus on the servers themselves.
$ sudo nix build /etc/nixos#nixosConfigurations.rasp-pi.config.system.build.toplevel --builders 'ssh-ng://rasp-pi aarch64-linux' --max-jobs 0 builders-use-substitutes --option builders-use-substitutes true
# relies on $(readlink ./result) already being in the rasp-pi's nix store from using it as a remote builder
# it would be nice if there was a way to avoid having to copy this back to the workstation machine
$ ssh root@rasp-pi nix-env -p /nix/var/nix/profiles/system --set $(readlink ./result)
$ ssh root@rasp-pi /nix/var/nix/profiles/system/bin/switch-to-configuration switch
Ah I see you’re building on a remote system with different architecture while I build on localhost an x86_64-linux system configurations for aarch64-linux systems.
error: build of '/nix/store/48x117xydlkij563h99xwk9npwl0ggc9-ensure-all-wrappers-paths-exist.drv' on 'ssh-ng://nixbuilder@radahn' failed: error: a 'aarch64-linux' with features {} is required to build '/nix/store/48x117xydlkij563h99xwk9npwl0ggc9-ensure-all-wrappers-paths-exist.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}
kind of suspect that I messed up with the system = ... – this has been quite confusing for me.
Thanks! I forgot to mention that the target system, though can be accessed via ssh, is not connected to internet. I assume --build-host would fail in this case?
You can add this line to your PC then you can build for aarch64 as well and push it to the Pi.
You then need to set —build-host localhost in order to build it on your PC.
Since we now have a binary cache for aarch64 the whole thing is quite fast.
In that case you could perhaps add it as a remote builder. IIRC remote builders always gather deps on the controlling machine and copy them over. This is annoying in most cases but probably what you want here.
Thanks. Actually had that line on my laptop, and running nixos-rebuild switch --target-host from the laptop does not work, error is still about not being able to build aarch64-linux.
Haven’t tried that from the pi side though, will give it a try.
what is interesting is that the documentation says:
--fast
Equivalent to --no-build-nix. This option is useful if you call nixos-rebuild frequently (e.g. if you’re hacking on a NixOS module).
where:
--no-build-nix
Normally, nixos-rebuild first builds the nixUnstable attribute in Nixpkgs, and uses the resulting instance of the Nix package manager to build the new system configuration. This is necessary if the NixOS modules use features not provided by the currently installed version of Nix. This option disables building a new Nix.
However running it with --no-build-nix instead of --fast fails as before: