Darwin.builder on an m1 mac problems with mkfs.ext4

Help
1

Hi, I’ve read @Gabriel439’s post Haskell for all: Nixpkgs support for Linux builders running on macOS and Nixpkgs 23.11 manual | Nix & NixOS and it seems very exciting. I’m hoping to automate building some nixos vms for people on my team both on mac and linux, and this seems a great way to get it working for everyone.

My understanding is that I should be able to run nix run nixpkgs#darwin.builder and it should launch a linux nixos vm for building using qemu all in one step.

This requires macOS version 12.4 or later.

I’m on 13.4

You will also need to be a trusted user for your Nix installation

I have the extra-trusted-users set up.

But after creating the ssh keys (first run only), it breaks:

❯ nix run nixpkgs#darwin.builder
Disk image do not exist, creating the virtualisation disk image...
Formatting '/tmp/tmp.PlKPnRQYvj', fmt=raw size=21474836480
/nix/store/4mx7h23lbrf1csjmnf5fcd9nvzgrya2n-nixos-vm/bin/run-nixos-vm: line 14: /nix/store/sh0plpi3ddar4g7ws7ww37r5kv0rg51j-e2fsprogs-1.47.0-bin/bin/mkfs.ext4: cannot execute binary file: Exec format error

appears like the run-nixos-vm script is trying to run linux binary

❯ file /nix/store/sh0plpi3ddar4g7ws7ww37r5kv0rg51j-e2fsprogs-1.47.0-bin/bin/mkfs.ext4
/nix/store/sh0plpi3ddar4g7ws7ww37r5kv0rg51j-e2fsprogs-1.47.0-bin/bin/mkfs.ext4: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/n8hgw8lja68x3zhv2ww9a7b6npbkvh21-glibc-2.37-8/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.10.0, not stripped

The failing part of the script /nix/store/4mx7h23lbrf1csjmnf5fcd9nvzgrya2n-nixos-vm/bin/run-nixos-vm is

   7   │ # Create an empty ext4 filesystem image. A filesystem image does not
   8   │ # contain a partition table but just a filesystem.
   9   │ createEmptyFilesystemImage() {
  10   │   local name=$1
  11   │   local size=$2
  12   │   local temp=$(mktemp)
  13   │   /nix/store/lyr3h4mi8nqhbk6gz08k6rq6wnq8rpy9-qemu-host-cpu-only-8.0.2/bin/qemu-img create -f raw "$temp" "$size"
  14   │   /nix/store/sh0plpi3ddar4g7ws7ww37r5kv0rg51j-e2fsprogs-1.47.0-bin/bin/mkfs.ext4 -L nixos "$temp"
  15   │   /nix/store/lyr3h4mi8nqhbk6gz08k6rq6wnq8rpy9-qemu-host-cpu-only-8.0.2/bin/qemu-img convert -f raw -O qcow2 "$temp" "$name"
  16   │   rm "$temp"
  17   │ }

And the other binaries being executed there look to be the correct format, just mkfs.ext4 is the linux one somehow.

❯ file /nix/store/lyr3h4mi8nqhbk6gz08k6rq6wnq8rpy9-qemu-host-cpu-only-8.0.2/bin/qemu-img
/nix/store/lyr3h4mi8nqhbk6gz08k6rq6wnq8rpy9-qemu-host-cpu-only-8.0.2/bin/qemu-img: Mach-O 64-bit executable arm64
❯ file /nix/store/sh0plpi3ddar4g7ws7ww37r5kv0rg51j-e2fsprogs-1.47.0-bin/bin/mkfs.ext4
/nix/store/sh0plpi3ddar4g7ws7ww37r5kv0rg51j-e2fsprogs-1.47.0-bin/bin/mkfs.ext4: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/n8hgw8lja68x3zhv2ww9a7b6npbkvh21-glibc-2.37-8/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.10.0, not stripped
❯ file /nix/store/lyr3h4mi8nqhbk6gz08k6rq6wnq8rpy9-qemu-host-cpu-only-8.0.2/bin/qemu-img
/nix/store/lyr3h4mi8nqhbk6gz08k6rq6wnq8rpy9-qemu-host-cpu-only-8.0.2/bin/qemu-img: Mach-O 64-bit executable arm64

Is this a regression maybe in nixpkgs-unstable, or could there be something wrong with my system? I’m not sure where to start debugging.

2

Since darwin.builder is in Nixpkgs, you should create an issue in Nixpkgs.

3

This has been reported here:

… and there is a related pull request open:

2 Likes
4

Can someone actually confirm this PR solves the issue? Neither OP neither me are using Darwin anyway.

5

I just confirmed that it solves the issue:

1 Like
6

Note that it still won’t work on x86_64-darwin immediately after that PR because e2fsprogs is broken there for unrelated reasons, but that’s being worked on currently. You can use nixpkgs/nixpkgs-23.05-darwin#darwin.builder in the meantime.