Remote building flake support on Darwin

Help
1

What is the correct command to be able to build a flake using a remote-builder?

I have an m1 mac with nax-darwin and docker desktop instaled. I have been experimenting
with setting up a docker container to use as a remote builder with limited success.

I can do the following to build for aarch64-linux using the remote builder successfully:

nix build --impure --expr '(import <nixpkgs> { system = "aarch64-linux"; }).hello' --builders 'ssh://nix-docker aarch64-linux'

But with a flake like the below

{
  inputs.nixpkgs.url = github:NixOS/nixpkgs/nixos-22.05;

  outputs = { self, nixpkgs }:
    let
      pkgs = nixpkgs.legacyPackages.aarch64-linux;
    in {
      packages.aarch64-linux.default = pkgs.hello;
    };
}

and running nix build --builders 'ssh://nix-docker aarch64-linux' I get this error:

error: flake 'path:/Users/geraint/Documents/work/nix_darwin_docker' does not provide attribute 'packages.aarch64-darwin.default' or 'defaultPackage.aarch64-darwin'

Am I missing something from my build command or flake, or is this not yet supported?

2

IDK about flakes but if you don’t need the result on the host’s store, use --store ssh-ng://nix-docker instead.

3

nix build uses your current system to lookup flake outputs, so it looks for packages.aarch64-darwin.default while you provide only packages.aarch64-linux.default. You can either specify full path to the output or pass --system aarch64-linux argument to nix build, then it will find and build the output that you provide.

1 Like
4

Be careful with --system aarch64-linux though. It will try to actually consider your local machine a linux system, which means that derivations with preferLocalBuild will try to build locally instead of on a builder, and fail. --max-jobs 0 should cut that out though

1 Like
5

I used --system flag like this without running into such issue. I think it won’t affect users running Nix daemon, only those who uses Nix in single-user mode.

6

It only happens for derivations with preferLocalBuild = true;

$ nix build --system x86_64-linux --impure --expr 'with import <nixpkgs> {}; runCommand "foo" { preferLocalBuild = true; } "touch $out"'
error: builder for '/nix/store/y69h2h703kwvryj28sn9mspr41sh7ncl-foo.drv' failed with exit code 126;
       last 1 log lines:
       > /nix/store/2r9n7fz1rxq088j6mi5s7izxdria6d5f-bash-5.1-p16/bin/bash: /nix/store/2r9n7fz1rxq088j6mi5s7izxdria6d5f-bash-5.1-p16/bin/bash: cannot execute binary file
       For full logs, run 'nix log /nix/store/y69h2h703kwvryj28sn9mspr41sh7ncl-foo.drv'.

Nix thinks it’s an x86_64-linux when it’s not, sees preferLocalBuild, and decides to try to build a linux derivation locally on a darwin machine.

7

Thanks all for your help! I tried nix build --system aarch64-linux --max-jobs 0 --builders 'nix-docker aarch64-linux' and it worked perfectly, allowing me to create docker images using flakes on nix-darwin :grinning_face_with_smiling_eyes:

I know I could put these flags in nix.conf to avoid having to write them out each time, but is there a way of specifying this per project (kind of like a local .nixrc file?)

8

You can add nixConfig attribute set in the top level of the flake that would contain additional configuration that will be used by Nix when running commands with it.

1 Like