ryang
November 17, 2022, 4:39pm
9
NB this doesn’t work for different architectures nixos-rebuild fails cross-build a flake · Issue #166499 · NixOS/nixpkgs · GitHub .
Adapted from @misuzu ’s answer, a workaround is:
$ sudo nix build /etc/nixos#nixosConfigurations.rasp-pi.config.system.build.toplevel --builders 'ssh-ng://rasp-pi aarch64-linux' --max-jobs 0 builders-use-substitutes --option builders-use-substitutes true
# relies on $(readlink ./result) already being in the rasp-pi's nix store from using it as a remote builder
# it would be nice if there was a way to avoid having to copy this back to the workstation machine
$ ssh root@rasp-pi nix-env -p /nix/var/nix/profiles/system --set $(readlink ./result)
$ ssh root@rasp-pi /nix/var/nix/profiles/system/bin/switch-to-configuration switch
opened 03:47PM - 09 Nov 22 UTC
closed 11:03AM - 10 Nov 22 UTC
0.kind: bug
### Describe the bug
`nix-rebuild` doesn't work for targets with a different … architecture
### Steps To Reproduce
Have a AArch64 Raspberry Pi and an x86_64 workstation.
Using the flake [RyanGibb/nixos](github.com/RyanGibb/nixos):
```
$ sudo nixos-rebuild switch --flake /etc/nixos#rasp-pi --build-host rasp-pi --target-host rasp-pi
error: a 'aarch64-linux' with features {} is required to build '/nix/store/9h2wxggmp52bzwshfjxygnq1nj3k1xbc-nixos-rebuild.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}
```
It does not seem to pick up `--build-host rasp-pi`.
If we set `builders`...
```
$ sudo nixos-rebuild switch --flake /etc/nixos#rasp-pi --build-host rasp-pi --target-host rasp-pi --builders 'ssh://rasp-pi aarch64-linux'
/nix/store/xcvbianp6ljciij03i7nhgiql0vz652a-nixos-rebuild/bin/nixos-rebuild: line 375: /nix/store/1gjq7k2a4ry774a8x9a8vgf9x3lflisk-coreutils-9.1/bin/mktemp: cannot execute binary file: Exec format error
```
I suspect the AArchf64 `mktemp` binary is being called on the NixOS machine. Using `--no-build-nix` does not make a difference.
### Expected behavior
The script to evaluate the Nix expression locally but build derivations remotely using the Pi, and deploy on the Pi.
### Screenshots
If applicable, add screenshots to help explain your problem.
### Additional context
Add any other context about the problem here.
### Notify maintainers
Not sure who maintains `nixos-rebuild`, but looking at https://github.com/NixOS/nixpkgs/commits/master/pkgs/os-specific/linux/nixos-rebuild, it seems @rnhmjoj and @jonringer have touched it recently.
### Metadata
```console
[workstation] $ nix-shell -p nix-info --run "nix-info -m"
- system: `"x86_64-linux"`
- host os: `Linux 5.15.75, NixOS, 22.11 (Raccoon), 22.11.20221031.d40fea9`
- multi-user?: `yes`
- sandbox: `yes`
- version: `nix-env (Nix) 2.11.0`
- channels(root): `"nixos"`
- channels(ryan): `""`
- nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
[workstation] $ nix-shell -p nix-info --run "nix-info -m"
- system: `"aarch64-linux"`
- host os: `Linux 5.15.75, NixOS, 22.11 (Raccoon), 22.11.20221031.d40fea9`
- multi-user?: `yes`
- sandbox: `yes`
- version: `nix-env (Nix) 2.11.0`
- channels(root): `"nixos-22.05"`
- nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
```