I’m trying to deploy a NixOS machine using nixops
on MacOS. I configured a x86_64-linux
build machine so that my nix on MacOS can build derivations for Linux:
cat /etc/nix/machines
ssh://nixBuild@X.X.X.X x86_64-linux /var/root/.ssh/id_nixBuild 48 2 benchmark,big-parallel,nixos-test
The NixOS configuration that I’m trying to deploy also contains the following to make sure all derivations are build for Linux:
nixpkgs.localSystem = { system = "x86_64-linux"; };
Let’s try to deploy it (note that the configuration also enables hydra):
$ nixops deploy ...
...
building '/nix/store/k1i3mnzl3rx66inm990xkm4hpdn5ndc4-hydra-0.1.2544.63a294.drv' on 'ssh://nixBuild@X.X.X.X'...
...
checking whether /nix/store/dhz34xy8k97i7551dhmmv9kbipd1gvji-hydra-perl-deps/bin/nix-store is recent enough... ./configure: line 16254: /nix/store/dhz34xy8k97i7551dhmmv9kbipd1gvji-hydra-perl-deps/bin/nix-store: cannot execute binary file: Exec format error
no
...
builder for '/nix/store/k1i3mnzl3rx66inm990xkm4hpdn5ndc4-hydra-0.1.2544.63a294.drv' failed with exit code 1
So Nix is correctly performing the build of the hydra derivation on my Linux build machine. However the build fails during the configure phase when executing the nix-store
executable. The reason is that this executable is build for MacOS and not for Linux. As in I can run it correctly from my Mac:
$ /nix/store/dhz34xy8k97i7551dhmmv9kbipd1gvji-hydra-perl-deps/bin/nix-store --version
nix-store (Nix) 2.1pre6377_954d1f4d
But if I execute the same on the NixOS build server:
$ ssh root@X.X.X.X /nix/store/dhz34xy8k97i7551dhmmv9kbipd1gvji-hydra-perl-deps/bin/nix-store --version
bash: /nix/store/dhz34xy8k97i7551dhmmv9kbipd1gvji-hydra-perl-deps/bin/nix-store: cannot execute binary file: Exec format error
Any idea why these Mac executables end up in the hydra derivation that is configured to be build for x86_64-linux
?
One suspicion I have is that this is caused by buildEnv
preferring local builds over using a remote builder. Note that the hydra derivation is using buildEnv
for building the problematic hydra-perl-deps
derivation.
I’m currently doing a build where I commented out preferLocalBuild = true
in buildenv/default.nix
to see if that fixes things…