Submitting a build to a builder and copying it to another server

Help
1

I have the following scenario: I have three machines: localhost, builder and target. I have a git flake that I evaluate on localhost that I want to build on builder but deploy on target. I’ve ended up creating a script that boils down to this:

drv=$(nix eval --raw ".#nixosConfigurations.target.config.system.build.toplevel.drvPath")
nix copy --to "ssh-ng://builder" "$drv"
output=$(ssh builder "nix build --print-out-paths --no-link $drv^out")
nix copy --from "ssh-ng://builder" --to "ssh-ng://target" "$output"
ssh target "nix build --profile /nix/var/nix/profiles/system $output"
ssh target "/nix/var/nix/profiles/system/bin/switch-to-configuration switch"

The reason I’m doing this instead of using a remote builder is so the whole build is done on the builder host without having to copy back and forth intermediate steps onto localhost (in this scenario, I don’t have a very stable internet connection). Target and builder also can’t reach each other because of networking constraints.

My question is: is there a nicer way to perform this pattern? E.g. I’d like to copy build results from the builder to localhost and onto target as they’re completed, but that doesn’t work right now.

2

To answer my own question: nixos-rebuild-ng basically does the same as my script, so it’s probably the best approach right now: https://github.com/NixOS/nixpkgs/tree/master/pkgs/by-name/ni/nixos-rebuild-ng

3

I use it exactly this way with --target-host and --build-host.

You use nixos-rebuild which defaults to nixos-rebuild-ng nowadays. It was working with the old one as well.

2 Likes
4

I have another version of this: instead of doing this:

output=$(ssh builder "nix build --print-out-paths --no-link $drv^out")

you can do this:

output=$(NIX_REMOTE="ssh-ng://builder" nix build --no-link --print-out-paths "$drv^out")

Which will use the nix remote builder protocol, making it easier to lock down as the SSH user then only needs access to nix-daemon --stdio and no other commands.

1 Like