Remote building flake support on Darwin

What is the correct command to be able to build a flake using a remote-builder?

I have an m1 mac with nax-darwin and docker desktop instaled. I have been experimenting
with setting up a docker container to use as a remote builder with limited success.

I can do the following to build for aarch64-linux using the remote builder successfully:

nix build --impure --expr '(import <nixpkgs> { system = "aarch64-linux"; }).hello' --builders 'ssh://nix-docker aarch64-linux'

But with a flake like the below

  inputs.nixpkgs.url = github:NixOS/nixpkgs/nixos-22.05;

  outputs = { self, nixpkgs }:
      pkgs = nixpkgs.legacyPackages.aarch64-linux;
    in {
      packages.aarch64-linux.default = pkgs.hello;

and running nix build --builders 'ssh://nix-docker aarch64-linux' I get this error:

error: flake 'path:/Users/geraint/Documents/work/nix_darwin_docker' does not provide attribute 'packages.aarch64-darwin.default' or 'defaultPackage.aarch64-darwin'

Am I missing something from my build command or flake, or is this not yet supported?

IDK about flakes but if you don’t need the result on the host’s store, use --store ssh-ng://nix-docker instead.

nix build uses your current system to lookup flake outputs, so it looks for packages.aarch64-darwin.default while you provide only packages.aarch64-linux.default. You can either specify full path to the output or pass --system aarch64-linux argument to nix build, then it will find and build the output that you provide.

1 Like

Be careful with --system aarch64-linux though. It will try to actually consider your local machine a linux system, which means that derivations with preferLocalBuild will try to build locally instead of on a builder, and fail. --max-jobs 0 should cut that out though

1 Like

I used --system flag like this without running into such issue. I think it won’t affect users running Nix daemon, only those who uses Nix in single-user mode.

It only happens for derivations with preferLocalBuild = true;

$ nix build --system x86_64-linux --impure --expr 'with import <nixpkgs> {}; runCommand "foo" { preferLocalBuild = true; } "touch $out"'
error: builder for '/nix/store/y69h2h703kwvryj28sn9mspr41sh7ncl-foo.drv' failed with exit code 126;
       last 1 log lines:
       > /nix/store/2r9n7fz1rxq088j6mi5s7izxdria6d5f-bash-5.1-p16/bin/bash: /nix/store/2r9n7fz1rxq088j6mi5s7izxdria6d5f-bash-5.1-p16/bin/bash: cannot execute binary file
       For full logs, run 'nix log /nix/store/y69h2h703kwvryj28sn9mspr41sh7ncl-foo.drv'.

Nix thinks it’s an x86_64-linux when it’s not, sees preferLocalBuild, and decides to try to build a linux derivation locally on a darwin machine.

Thanks all for your help! I tried nix build --system aarch64-linux --max-jobs 0 --builders 'nix-docker aarch64-linux' and it worked perfectly, allowing me to create docker images using flakes on nix-darwin :grinning_face_with_smiling_eyes:

I know I could put these flags in nix.conf to avoid having to write them out each time, but is there a way of specifying this per project (kind of like a local .nixrc file?)

You can add nixConfig attribute set in the top level of the flake that would contain additional configuration that will be used by Nix when running commands with it.

1 Like