Cross compiling docker images with flakes

Help
1

I’ve used regular old non-flake Nix in the past to “cross-compile” docker images. My local machine is macOS M1, and I can either override system (ie. import <nixpkgs> { system = "x86_64-linux"; }; or set architecture when building the docker image.

But with flakes, how would I define a flake that builds a docker image that runs on linux?

Do I use packages.x86_64-linux.containerImage = dockerTools.buildLayeredImage … because I want it to run on linux, or do I use packages.aarch64-darwin.containerImage = … because I’m building on a mac M1?

I have a remote builder available for linux x86_64…

1 Like
2

You can try native compilation to x86_64-linux using latest Docker Desktop with Rosetta emulation enabled:

First create a nix-docker container (current directory should be in the root of your flake):

if ! docker container inspect nix-docker > /dev/null 2>&1; then
  docker create --platform linux/amd64 --privileged --name nix-docker -it -w /work -v $(pwd):/work nixos/nix
  docker start nix-docker > /dev/null
  docker exec nix-docker bash -c "git config --global --add safe.directory /work"
  docker exec nix-docker bash -c "echo 'sandbox = true' >> /etc/nix/nix.conf"
  docker exec nix-docker bash -c "echo 'filter-syscalls = false' >> /etc/nix/nix.conf"
  docker exec nix-docker bash -c "echo 'max-jobs = auto' >> /etc/nix/nix.conf"
  docker exec nix-docker bash -c "echo 'experimental-features = nix-command flakes' >> /etc/nix/nix.conf"
fi

Then use nix-docker container to build your flake:

docker start nix-docker > /dev/null # start nix-docker container if not already running
docker exec -it nix-docker nix build -L .#containerImage
docker exec -it nix-docker cp ./result/* . # copy artifacts to the host system

You can automate these steps using bash script.

3

The whole reason why I’m trying Nix is to remove dependency on Docker :wink:

3 Likes
4

I figured that out today. If you build locally, you “use packages.aarch64-darwin.containerImage = … because I’m you are building on a mac M1” with import <nixpkgs> { localSystem = "aarch64-darwin", crossSystem = "x86_64-linux"; };.

Here an example (without fanciness), ./docker.nix would contain the dockerTools.buildLayeredImage derivation:

{
  outputs = { self, nixpkgs }: {
    # Regular build
    # packages.${localsystem}.${name} = drv { import pkgs = nixpkgs.legacypackages.${localsystem}; }
    packages.aarch64-darwin.containerImage = import ./docker.nix { pkgs = nixpkgs.legacyPackages.aarch64-darwin; };
    # Cross build
    # packages.${localSystem}."${name}-${crossSystem}" = drv { import nixpkgs { localSystem = localSystem; crossSystem = crossSystem;}
    packages.aarch64-darwin.containerImage-x86_64-linux = import ./docker.nix { pkgs = import nixpkgs { localSystem = "aarch64-darwin"; crossSystem = "x86_64-linux"; }; };
  };
}

For fancy generation, there is this post: How do I cross-compile a flake? - #22 by reckenrode and GitHub - numtide/flake-utils: Pure Nix flake utility functions [maintainer=@zimbatm]

Those are the conventions I have observed. For example: https://github.com/NixOS/nix/blob/435a16b5556f4171b4204a3f65c9dedf215f168c/flake.nix#L422-L423

3 Likes