Building a docker image on macOS with remote builder

Learn
#1

buildImage doesn’t use Docker itself and just adds to an image whatever has been built on host system, which means that if I’m building it on macOS - despite my building suceeding I won’t be able to run the image because executable format will be wrong:

$ docker run $SOME_IMAGE
standard_init_linux.go:211: exec user process caused "exec format error"

As as workaround I tried to use remote builder on top of LnL7/nix-docker image. Remote builder works as expected for nix-build <nixpkgs>:

$ nix-build '<nixpkgs>' -A hello --argstr system x86_64-linux
/nix/store/hs6rg4zbsclx660s6i5938605zmv6lgh-hello-2.10
$ ./result/bin/hello # error as expected
zsh: exec format error: ./result/bin/hello

But when I try to use it for my own nix expression, it still builds the macOS executable:

$ cat docker.nix
{ pkgs ? import <nixpkgs> { } }:
with pkgs;
dockerTools.buildImage {
  name = "nix-docker-test";
  tag = "0.0.1";
  contents = [ hello ];
  config = {
    Cmd = [ hello ];
    Entrypoint = [ stdenv.shell ];
  };
}
$ nix-build docker.nix --argstr system x86_64-linux
...
$ docker run $ANOTHER_IMAGE
standard_init_linux.go:211: exec user process caused "exec format error"

How do tell nix-build to use remote builder in this case? I tried to overwrite { system = "x86_64-linux" } in <nixpkgs>, but got:

error: a 'x86_64-linux' with features {} is required to build '/nix/store/i3bv2aghqy4ib5xczlm3wv6r39vpdlj4-nix-docker-test-config.json.drv', but I am a 'x86_64-darwin' with features {benchmark, big-parallel, nixos-test}
#2

Try updating your source file like so to specify the target system type appropriately:

{ system ? "x86_64-linux", pkgs ? import <nixpkgs> { inherit system; } }:
with pkgs;
# ...
1 Like
#3

Nope, didn’t work. Still the same

error: a 'x86_64-linux' with features {} is required to build '/nix/store/i3bv2aghqy4ib5xczlm3wv6r39vpdlj4-nix-docker-test-config.json.drv', but I am a 'x86_64-darwin' with features {benchmark, big-parallel, nixos-test}

Last time I did just:

{ pkgs ? import <nixpkgs> { system = "x86_64-linux"; } }

I think, result is the same?

#4

Ok, I think it’s something weird going on with my SSH connection. I tried to force it to use remote builders only by:

$ nix-build docker.nix --max-jobs 0 --builders nix-docker -vvvvvvvvv 2>&1

I was getting different results depending on some other minor parameters (lost track of them), but it does try to build on remote host and eventually I get

hook reply is 'decline'

If I ssh into the container and try to install something, I get:

$ ssh nix-docker
lastlog_openseek: Couldn't stat /var/log/lastlog: No such file or directory
lastlog_openseek: Couldn't stat /var/log/lastlog: No such file or directory
-bash-4.4# nix-env -i hello
warning: there are multiple derivations named 'hello-2.10'; using the first one
installing 'hello-2.10'
warning: unable to download 'https://cache.nixos.org/kmnqp3hvhz7ym9inm4m4bnkmmwxc6mzz.narinfo': SSL peer certificate or SSH remote key was not OK (60); retrying in 305 ms
warning: unable to download 'https://cache.nixos.org/kmnqp3hvhz7ym9inm4m4bnkmmwxc6mzz.narinfo': SSL peer certificate or SSH remote key was not OK (60); retrying in 699 ms
warning: unable to download 'https://cache.nixos.org/kmnqp3hvhz7ym9inm4m4bnkmmwxc6mzz.narinfo': SSL peer certificate or SSH remote key was not OK (60); retrying in 1019 ms
warning: unable to download 'https://cache.nixos.org/kmnqp3hvhz7ym9inm4m4bnkmmwxc6mzz.narinfo': SSL peer certificate or SSH remote key was not OK (60); retrying in 2376 ms
error: unable to download 'https://cache.nixos.org/kmnqp3hvhz7ym9inm4m4bnkmmwxc6mzz.narinfo': SSL peer certificate or SSH remote key was not OK (60)

But what is weirder, if I just login to the container by docker exec -it $CONT /bin/sh - everything works just fine. So I think some SSL config just gets propagated from my host machine.

Hosted by Flying Circus.