Deploy NixOS configurations on other Machines

Help
21

it doesn’t work for me when the target architecture is different than the host architecture that calls the command.

This one works as it targets the same arch - x86_64-linux

nixos-rebuild switch --flake .#deckard --target-host deckard --use-remote-sudo

Now, a similar command but I am targeting an aarch64 machine and using nixbuild builder (this one also works):

nixos-rebuild build --max-jobs 0  --build-host "eu.nixbuild.net" --target-host surfer --use-remote-sudo  --flake .#surfer

However if I replace build with switch the command fails:

$ nixos-rebuild switch --max-jobs 0  --build-host "eu.nixbuild.net" --target-host surfer --use-remote-sudo  --flake .#surfer
/nix/store/5yksn2xwy3aif5pxz353i64i0fwvj5gp-nixos-rebuild/bin/nixos-rebuild: line 382: /nix/store/zlh7zakv2fn97fb4q2y6abzsdp6jflfd-coreutils-9.3/bin/mktemp: cannot execute binary file: Exec format error

Any idea what could be wrong?

22

Did you declare the hostSystem in surfer’s config?

23

No, I only have system = "aarch64-linux. I thought that hostSystem property is only required when doing some cross-compilation, isn’t it?

I kind of solved my problem, i.e. I found the correct set of flags that work but I would like to understand what is going on.

The correct command looks as follow:

nixos-rebuild --max-jobs 0  --builders "ssh://eu.nixbuild.net aarch64-linux - 100 1" --flake .#surfer --target-host surfer --fast --use-remote-sudo switch

what is interesting is that the documentation says:

--fast
Equivalent to --no-build-nix. This option is useful if you call nixos-rebuild frequently (e.g. if you’re hacking on a NixOS module).

where:

--no-build-nix
Normally, nixos-rebuild first builds the nixUnstable attribute in Nixpkgs, and uses the resulting instance of the Nix package manager to build the new system configuration. This is necessary if the NixOS modules use features not provided by the currently installed version of Nix. This option disables building a new Nix.

However running it with --no-build-nix instead of --fast fails as before:

 nixos-rebuild --no-build-nix --max-jobs 0  --builders "ssh://eu.nixbuild.net aarch64-linux - 100 1" --flake .#surfer --target-host surfer switch
/nix/store/5yksn2xwy3aif5pxz353i64i0fwvj5gp-nixos-rebuild/bin/nixos-rebuild: line 382: /nix/store/zlh7zakv2fn97fb4q2y6abzsdp6jflfd-coreutils-9.3/bin/mktemp: cannot execute binary file: Exec format error
24

What was this pointing at before? Right now the line is just a closing bracket.

25

Right, forget to make it a permalink. It would be this line here: nixos/systems/gwyn/default.nix at dac50a1ab18bbf5628a6bcac96902d70da84dcb7 · Nebucatnetzer/nixos · GitHub

1 Like
26

Thanks! Of course, you need to be able to build on your own machine.

27

When --build-host myremote is given, what channel (nixos version) is used? The local or the remote one?

28

Eval happens locally on the machine running nixos-rebuild, so that machine’s channel.

29

Thx. Well, that can be annoying when I have auto update enabled on the remote host and the remote channel is more up to date than the local one.

Can I maintain a local channel just for the remote build eval independent from the channel used for my local configuration?

30

I don’t see an obvious solution to this unfortunately.

Sure. You’d have to pass -I nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs-for-machine to nixos-rebuild every time though.

1 Like
31

I was getting this error while doing the same thing on a Github runner (x86_64-linux building for aarch64-linux):

bin/mktemp: cannot execute binary file: Exec format error

…and of course changing boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; in a Github runner is impractical, but it turns out that with QEMU installed and extra-platforms set, it works for me:

      - name: Install QEMU
        uses: docker/setup-qemu-action@v3

      - name: Install nix
        uses: cachix/install-nix-action@v27
        with:
          enable_kvm: true
          extra_nix_config: |
            system-features = kvm
            extra-platforms = aarch64-linux

Hope this helps those who come next.

32

Hey,

Just wanted to let you know that I was also getting the following error when running nixos-rebuild from Apple silicon aarch64-darwin with virtual x86_64-linux capable builder:

/nix/.../coreutils-9.5/bin/mktemp: cannot execute binary file: Exec format error

But when I also added --fast flag for the nixos-rebuild command this started to work for me :+1: