Failed to build cross-architecture NixOS system image

I’m trying to build a NixOS system image on x86_64-linux using this flake.nix:

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
  outputs = { self, nixpkgs }: {
    packages.x86_64-linux.default = self.nixosConfigurations.image.config.system.build.images.raw-efi;
    nixosConfigurations.image = nixpkgs.lib.nixosSystem {
      modules = [ { nixpkgs.hostPlatform.system = "aarch64-linux"; } ];
    };
  };
}

It works fine when nixpkgs.hostPlatform.system is set to x86_64-linux — the expected .img file is produced. However, after I added boot.binfmt.emulatedSystems = [ "aarch64-linux" ] in my /etc/nixos/configuration.nix and switched nixpkgs.hostPlatform.system to aarch64-linux, running nix build -L . spews out a large number of errors like:

nixos-disk-image> copying path '/nix/store/7llyx0szys4k1jaapd6jjcf0j3lcfdb3-systemd-boot' to 'local'...
nixos-disk-image> copying path '/nix/store/09g7m9vpkdyq3hr7q4kzd03qif0dcy2i-install-systemd-boot.sh' to 'local'...
nixos-disk-image> copying path '/nix/store/dkkdq2ym3yzlb2njdlyl6byn2n6zhalp-unit-script-growpart-start' to 'local'...
nixos-disk-image> copying path '/nix/store/lmvdaqsksx3mjgzwpyr9k57h3c47b98h-unit-growpart.service' to 'local'...
nixos-disk-image> copying path '/nix/store/zjwppzaqfsd42q43sgycvicfn2p5vm2i-system-path' to 'local'...
nixos-disk-image> copying path '/nix/store/smp2h6wk109sbq2g0dnap1rx589ws36k-linux-6.18.40-modules' to 'local'...
nixos-disk-image> copying path '/nix/store/ky5266j9ykiqjx1yjf41ls5xx1dgfy6p-dbus-1' to 'local'...
nixos-disk-image> copying path '/nix/store/l0vij21k3kcpjz35ywi73hqr42cv1m34-X-Restart-Triggers-dbus-broker' to 'local'...
nixos-disk-image> copying path '/nix/store/ymmpsfcix67fbqn9g8694hrfh300wjc3-unit-dbus-broker.service' to 'local'...
nixos-disk-image> copying path '/nix/store/knc30s8r3r9i9n2jsksb8ak4v1hf0yi8-unit-dbus-broker.service' to 'local'...
nixos-disk-image> copying path '/nix/store/d4i4m5jw8f5sxr7jrq8mpw8jkdldklb9-user-units' to 'local'...
nixos-disk-image> copying path '/nix/store/8ss4mciz3vi1dvwlw560qiw3908fp0yw-system-units' to 'local'...
nixos-disk-image> copying path '/nix/store/vhlmqq8abvyx32srx8rb6njmvjkjfc3r-etc-nix-registry.json' to 'local'...
nixos-disk-image> copying path '/nix/store/gzhmlmziv9w0glpmhm19pvrv4iqmvz9p-etc' to 'local'...
nixos-disk-image> copying path '/nix/store/kpybhr610ym3b13lfjpqbv6pnyfyaaa1-nixos-system-nixos-efi-raw-26.11.20260726.624af66' to 'local'...
nixos-disk-image> installation finished!
nixos-disk-image> Automatic disk size...
nixos-disk-image>   Closure space use: 4129173504 bytes
nixos-disk-image>   fudge: 214717022 bytes
nixos-disk-image>   Filesystem size needed: 4343894016 bytes
nixos-disk-image>   Additional space: 806372352 bytes
nixos-disk-image>   Disk image size: 5150605312 bytes
nixos-disk-image> Model:  (file)
nixos-disk-image> Disk /build/nixos.raw: 5151MB
nixos-disk-image> Sector size (logical/physical): 512B/512B
nixos-disk-image> Partition Table: gpt
nixos-disk-image> Disk Flags:
nixos-disk-image>
nixos-disk-image> Number  Start   End     Size    File system  Name     Flags
nixos-disk-image>  1      8389kB  269MB   261MB   fat32        ESP      boot, esp
nixos-disk-image>  2      269MB   5150MB  4880MB  ext4         primary
nixos-disk-image>
nixos-disk-image> Warning: The kernel is still using the old partition table.
nixos-disk-image> The new table will be used at the next reboot or after you
nixos-disk-image> run partprobe(8) or kpartx(8)
nixos-disk-image> The operation has completed successfully.
nixos-disk-image> mke2fs 1.47.4 (6-Mar-2025)
nixos-disk-image> Discarding device blocks: done
nixos-disk-image> Creating filesystem with 1191424 4k blocks and 298368 inodes
nixos-disk-image> Filesystem UUID: 2a3889c9-9f48-4182-9134-98c9e572b234
nixos-disk-image> Superblock backups stored on blocks:
nixos-disk-image>       32768, 98304, 163840, 229376, 294912, 819200, 884736
nixos-disk-image>
nixos-disk-image> Allocating group tables: done
nixos-disk-image> Writing inode tables: done
nixos-disk-image> Creating journal (16384 blocks): done
nixos-disk-image> Writing superblocks and filesystem accounting information: done
nixos-disk-image>
nixos-disk-image> copying staging root to image...
nixos-disk-image> [    0.000000] Linux version 6.6.0 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1 Sat Jul 18 01:32:10 UTC 2026
nixos-disk-image> [    0.000000] memblock address range: 0x7fffd5c00000 - 0x7fffdc000000
nixos-disk-image> [    0.000000] Zone ranges:
nixos-disk-image> [    0.000000]   Normal   [mem 0x00007fffd5c00000-0x00007fffdbffffff]
nixos-disk-image> [    0.000000] Movable zone start for each node
nixos-disk-image> [    0.000000] Early memory node ranges
nixos-disk-image> [    0.000000]   node   0: [mem 0x00007fffd5c00000-0x00007fffdbffffff]
nixos-disk-image> [    0.000000] Initmem setup node 0 [mem 0x00007fffd5c00000-0x00007fffdbffffff]
nixos-disk-image> [    0.000000] Kernel command line:  mem=100M virtio_mmio.device=352@0x1000000:1
nixos-disk-image> [    0.000000] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
nixos-disk-image> [    0.000000] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
nixos-disk-image> [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 25250
nixos-disk-image> [    0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
nixos-disk-image> [    0.000000] Memory: 100768K/102400K available (6898K kernel code, 1663K rwdata, 1564K rodata, 139K init, 337K bss, 1632K reserved, 0K cma-reserved)
nixos-disk-image> [    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
nixos-disk-image> [    0.000000] trace event string verifier disabled
nixos-disk-image> [    0.000000] NR_IRQS: 4096
nixos-disk-image> [    0.000000] lkl: irqs initialized
nixos-disk-image> [    0.000000] clocksource: lkl: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
nixos-disk-image> [    0.000029] lkl: time and timers initialized (irq2)
nixos-disk-image> [    0.001607] pid_max: default: 4096 minimum: 301
nixos-disk-image> [    0.005436] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
nixos-disk-image> [    0.005534] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
nixos-disk-image> [    0.094996] printk: console [lkl_console0] enabled
nixos-disk-image> [    0.097354] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
nixos-disk-image> [    0.101053] NET: Registered PF_NETLINK/PF_ROUTE protocol family
nixos-disk-image> [    0.107371] lkl_pci: probe of lkl_pci failed with error -1
nixos-disk-image> [    0.270921] raid6: skipped pq benchmark and selected int64x8
nixos-disk-image> [    0.271032] raid6: using intx1 recovery algorithm
nixos-disk-image> [    0.288161] vgaarb: loaded
nixos-disk-image> [    0.289600] clocksource: Switched to clocksource lkl
nixos-disk-image> [    0.324391] NET: Registered PF_INET protocol family
nixos-disk-image> [    0.326614] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
nixos-disk-image> [    0.332062] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 4096 bytes, linear)
nixos-disk-image> [    0.332607] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
nixos-disk-image> [    0.332844] TCP established hash table entries: 1024 (order: 1, 8192 bytes, linear)
nixos-disk-image> [    0.333539] TCP bind hash table entries: 1024 (order: 2, 16384 bytes, linear)
nixos-disk-image> [    0.333878] TCP: Hash tables configured (established 1024 bind 1024)
nixos-disk-image> [    0.336279] UDP hash table entries: 128 (order: 0, 4096 bytes, linear)
nixos-disk-image> [    0.336644] UDP-Lite hash table entries: 128 (order: 0, 4096 bytes, linear)
nixos-disk-image> [    0.338863] PCI: CLS 0 bytes, default 32
nixos-disk-image> [    0.343990] virtio-mmio: Registering device virtio-mmio.0 at 0x1000000-0x100015f, IRQ 1.
nixos-disk-image> [    0.385141] workingset: timestamp_bits=62 max_order=15 bucket_order=0
nixos-disk-image> [    0.390314] SGI XFS with ACLs, security attributes, no debug enabled
nixos-disk-image> [    0.400101] xor: automatically using best checksumming function   8regs
nixos-disk-image> [    0.400616] io scheduler mq-deadline registered
nixos-disk-image> [    0.400854] io scheduler kyber registered
nixos-disk-image> [    0.462964] virtio_blk virtio0: 1/0/0 default/read/poll queues
nixos-disk-image> [    0.471426] virtio_blk virtio0: [vda] 10059776 512-byte logical blocks (5.15 GB/4.80 GiB)
nixos-disk-image> [    0.561423]  vda: vda1 vda2
nixos-disk-image> [    0.584076] NET: Registered PF_INET6 protocol family
nixos-disk-image> [    0.597999] Segment Routing with IPv6
nixos-disk-image> [    0.598391] In-situ OAM (IOAM) with IPv6
nixos-disk-image> [    0.599172] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
nixos-disk-image> [    0.694428] Btrfs loaded, zoned=no, fsverity=no
nixos-disk-image> [    0.700489] Warning: unable to open an initial console.
nixos-disk-image> [    0.701345] This architecture does not have kernel memory protection.
nixos-disk-image> [    0.701591] Run /init as init process
nixos-disk-image> [    0.838581] EXT4-fs (vda2): mounted filesystem 2a3889c9-9f48-4182-9134-98c9e572b234 r/w with ordered data mode. Quota mode: disabled.
nixos-disk-image> error while reading directory /build/root/etc: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var/nix/db: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var/nix/temproots: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var/nix/gcroots/auto: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var/nix/gcroots/per-user: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var/nix/gcroots: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var/nix/profiles/per-user/root: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var/nix/profiles/per-user: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var/nix/profiles: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var/nix: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/var: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/store/z6b3si9n21mqaapyjbwfhckhw98yz7np-shadow-4.19.4/lib: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/store/z6b3si9n21mqaapyjbwfhckhw98yz7np-shadow-4.19.4/bin: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/store/z6b3si9n21mqaapyjbwfhckhw98yz7np-shadow-4.19.4/share/locale/pl/LC_MESSAGES: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/store/z6b3si9n21mqaapyjbwfhckhw98yz7np-shadow-4.19.4/share/locale/pl: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/store/z6b3si9n21mqaapyjbwfhckhw98yz7np-shadow-4.19.4/share/locale/nl/LC_MESSAGES: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/store/z6b3si9n21mqaapyjbwfhckhw98yz7np-shadow-4.19.4/share/locale/nl: Cannot allocate memory
nixos-disk-image> error while reading directory /build/root/nix/store/z6b3si9n21mqaapyjbwfhckhw98yz7np-shadow-4.19.4/share/locale/ka/LC_MESSAGES: Cannot allocate memory
...

My machine still has plenty of free memory. Any idea what’s going on and how to fix this?

It turns out that setting boot.binfmt.emulatedSystems = [ "aarch64-linux" ] doesn’t work well for software emulation. I dug into the source code and found that the issue lies with the cptofs command. So it seems you really do need to run make-disk-image.nix on an actual arm64 device. That said, you could also patch the relevant source code and use something like mke2fs to achieve the same functionality.

I would generally recommend just doing actual cross compilation over doing binfmt.emulatedSystems. Needs no host configuration and is like 10x faster, at the expense of very little being cached.

Hmm… if it’s just building a few packages, sure, that works. But when it comes to building a full system image like I mentioned, a cache miss means I’d have to rebuild the entire system from scratch. :frowning: