How to build nixos system remotely?

I have an aarch64 box (machine A) which already run nixos. Initially I built the os locally with ‘nixos-rebuild’, but I now want to build from my beefy x86_64 desktop (machine B) in a bare-bone manner. However switch-to-configuration doesn’t seem to install the new system to grub menu.

Here’s what I did:

  • set up ssh and trusted-public-keys in machine A by running nixos-rebuild locally
  • nix build then nix-copy-closure --to user@machine-a ./result in machine B.
  • because ./result is a symlink to /nix/store/xxx-nixos-system-machine-a-xxx, in machine A, I ran /nix/store/xxx-nixos-system-machine-a-xxx/bin/switch-to-configuration switch

The new system (with defined packages and services) does seem to be activated in machine A, just grub menu is not updated. switch-to-configuration boot seems to have no effect. Did I miss something, or should I use another command?

I built with flake and a recent nixpkgs revision (Feb 22 - f5dad40450d272a1ea2413f4a67ac08760649e89), btw.

You can do remote activation (after the copying the closure or without copying first):
nixos-rebuild boot --target-host root@$machine-a --build-host localhost --flake some-flake#machine-a

I use a similar workflow (pre-build, copy, activate) for some laptops that are turned off most of the time.

Does that work? It seems that you’d build the system configuration for x86_64, and then that of course won’t work on aarch64?

If it’s built with flakes, then the system architecture won’t be implicit like that, so it will build for the right architecture. Now, how it builds for that architecture is up to @myguidingstar. I assume they have emulation set up with boot.binfmt.emulatedSystems (which, btw, is incredibly slow; a 16c/32t x86_64 processor emulating aarch64 isn’t much faster at compiling the kernel than a raspberry pi 4).

1 Like

yeah, I have the emulation enabled, just have problem with boot menu.

So I ended up with remote nixos-rebuild with remote root user (unable to sudo remotely is a known issue), similar to what @wamserma suggested.

Reading nixos-rebuild perl source code (a bit cryptic for me) suggested that the symlinks must be managed with nix-env before calling switch-to-configuration.