How to build nixos system remotely?

Help
1

I have an aarch64 box (machine A) which already run nixos. Initially I built the os locally with ‘nixos-rebuild’, but I now want to build from my beefy x86_64 desktop (machine B) in a bare-bone manner. However switch-to-configuration doesn’t seem to install the new system to grub menu.

Here’s what I did:

  • set up ssh and trusted-public-keys in machine A by running nixos-rebuild locally
  • nix build some-flake#nixosConfigurations.machine-a.config.system.build.toplevel then nix-copy-closure --to user@machine-a ./result in machine B.
  • because ./result is a symlink to /nix/store/xxx-nixos-system-machine-a-xxx, in machine A, I ran /nix/store/xxx-nixos-system-machine-a-xxx/bin/switch-to-configuration switch

The new system (with defined packages and services) does seem to be activated in machine A, just grub menu is not updated. switch-to-configuration boot seems to have no effect. Did I miss something, or should I use another command?

I built with flake and a recent nixpkgs revision (Feb 22 - f5dad40450d272a1ea2413f4a67ac08760649e89), btw.

2

You can do remote activation (after the copying the closure or without copying first):
nixos-rebuild boot --target-host root@$machine-a --build-host localhost --flake some-flake#machine-a

I use a similar workflow (pre-build, copy, activate) for some laptops that are turned off most of the time.

3

Does that work? It seems that you’d build the system configuration for x86_64, and then that of course won’t work on aarch64?

4

If it’s built with flakes, then the system architecture won’t be implicit like that, so it will build for the right architecture. Now, how it builds for that architecture is up to @myguidingstar. I assume they have emulation set up with boot.binfmt.emulatedSystems (which, btw, is incredibly slow; a 16c/32t x86_64 processor emulating aarch64 isn’t much faster at compiling the kernel than a raspberry pi 4).

1 Like
5

yeah, I have the emulation enabled, just have problem with boot menu.

So I ended up with remote nixos-rebuild with remote root user (unable to sudo remotely is a known issue), similar to what @wamserma suggested.

Reading nixos-rebuild perl source code (a bit cryptic for me) suggested that the symlinks must be managed with nix-env before calling switch-to-configuration.