Docker ignoring platform when run in NixOS

1

I’m runnning NixOS in a VM on an M1 mac. My application runs via docker compose.

One of the docker services is a mhart/alpine-node:14. When I run docker compose natively on macOS for my application, everything runs fine.

If I do the same in the nixOS VM, the docker mhart/alpine-node:14 image doesn’t run on the right platform and I get error:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
exec /usr/bin/npm: exec format error

I can’t figure out why docker behaves differently wrt to platforms when run in Nix.

2

What is the nature of the VM? How are you running it?

3

Vmware fusion tech preview. Used https://github.com/mitchellh/nixos-config as a starting point.

4

Is uname -m returning aarch64 in the nixos VM?

5

Yes it is. I assume docker compose is behaving different in nixos than on native mac. Platform is not being set somehow in nixos, my best guess.

6

I actually ran docker build on native mac and passed --platform linux/aarch64 and it worked fine but it failed on nixos when trying the same thing.

7

Docker desktop for mac includes x86 emulation. Are you sure the image isn’t actually an x86 one in both cases?

8

Firstly just wanna say thanks for replying everyone.

The image is x86 but I’ve run the same stack on my Ubuntu desktop and I didn’t have this problem. Docker-compose I assume passes platform target automatically, but somehow that isn’t happening in nix ? Not sure.

9

Ubuntu desktop’s dockerd might have a default platform set for the daemon.

10

Right now I am trying to figure out qemu emulation in my nixos vm. I’ve come to the point where running
docker buildx inspect --bootstrap
returns

 docker buildx inspect --bootstrap                                                                                  107ms  Sat 27 Aug 2022 07:39:57 AM PDT
[+] Building 0.5s (1/1) FINISHED                                                                                                                               
 => [internal] booting buildkit                                                                                                                           0.5s
 => => starting container buildx_buildkit_bldr0                                                                                                           0.5s
Name:   bldr
Driver: docker-container

Nodes:
Name:      bldr0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Platforms: linux/arm64

So my nixos isn’t capable of building for other platforms. I’m not sure how exactly to track down what’s missing but I am following this post to see if I can get a clue for now…

11

I have no idea about docker’s buildx but be aware than NixOS has boot.binfmt.emulatedSystems.

12

I’m assuming buildx is getting it’s platform emulation via binfmt. But even after adding
boot.binfmt.emulatedSystems= ["i386-linux" "x86_64-linux" ];
to my config, I still don’t get those platforms available to buildx builder.

From Docker docs multi arch:

Docker Desktop* provides binfmt_misc multi-architecture support, which means you can run
containers for different Linux architectures such as arm , mips , ppc64le , and even s390x .
This does not require any special configuration in the container itself as it uses qemu-static from the Docker for Mac VM . Because of this, you can run an ARM
container, like the arm32v7 or ppc64le variants of the busybox image.

13

Have you verified that you are able to execute x86 programs outside Docker?

14

So the entire problem was that my system didn’t have the emulators for other platforms somehow.

In all the docker buildx tutorials you see out there, people had cross platform emulation out of the box but my system only showed linux/arm64 when I ran docker buildx inspect .
Until I found this Repo. I ran docker run --privileged --rm tonistiigi/binfmt --install all just to make sure and finally my platform option in docker-compose or docker build worked.

So if anyone is stuck with the same problem, check what you get under the platforms column when you run docker buildx ls. If there’s only your own platform, you need to add the qemu emulators manually or via this docker command from the above linked repo. Thanks for your help @Atemu

2 Likes
15

just want to say thanks for documenting this. It solved my problem :slight_smile:
I was curious if there is some other way to do it instead of relying on a image