Turing Pi 2 + CM4

Hi,

I received my Turing Pi 2 a couple of weeks ago. Waited for a mini ITX case to arrive (already hoarded CM4 modules) and it works great.

First I installed Ubuntu 22.10 on each of my 4 CM4s and this worked. But then I said to myself: why not use this opportunity to learn NixOS better? So I switched to NixOS.

The CM4s I use are 2x 4 GB RAM 8 GB eMMC no WLAN and 2x 4 GB RAM 32 GB eMMC. I’m booting from eMMC.

I’ve tried with a recent Hydra AArch64 image as well as cross-compiling my own image using Qemu on x86-64. The images work perfectly well on every CM4, except for whichever CM4 is in slot #3. So if I swap them around, they always work well in slots #1 #2 and #4 but not in #3 (Ubuntu 22.10 works here).

There’s nothing useful in UART. I mean, all I see is uboot (normally it gets further and I see the OS booting up). It doesn’t get further. So I’ve been wondering, what could it be? It seems to be a boot loop (snippet @ [1]). Looking at Specs and I/O Ports what stands out is the third module has two SATA ports. But none of these are connected, and the UUID defined by NixOS configuration is unique.

[1] U-Boot 2022.10 (Oct 03 2022 - 19:25:32 +0000)\r\n\r\nDRAM: 3.9 GiB\r\nRPI Compute Module 4 (0xc03141)\r\nCore: 209 devices, 16 uclasses, devicetree: board\r\nMMC: mmcnr@7e300000: 1, mmc@7e340000: 0\r\nLoading Environment from FAT… Unable to read "uboot.env" from mmc0:1… \r\nIn: serial\r\nOut: serial\r\nErr: serial\r\nNet: eth0: ethernet@7d580000\r\nPCIe BRCM: link up, 5.0 Gbps x1 (SSC)\r\n"Error" handler, esr 0xbf000002\r\nelr: 00000000000b107c lr : 00000000000b1020 (reloc)\r\nelr: 000000003b37e07c lr : 000000003b37e020\r\nx0 : 000000000000dead x1 : 00000000fd508000\r\nx2 : 00000000fd508000 x3 : 0000000000000000\r\nx4 : 000000003af3b2d0 x5 : 0000000000000001\r\nx6 : 000000003b37f984 x7 : 0000000000000002\r\nx8 : 000000003b37fa60 x9 : 0000000000000008\r\nx10: 0000000000008a98 x11: 000000003af3b20c\r\nx12: 0000000000000140 x13: 000000003af3b1f8\r\nx14: 000000003af3b890 x15: 0000000000000020\r\nx16: 000000003b37fa60 x17: 0000000000000000\r\nx18: 000000003af48d70 x19: 000000003af3b2d0\r\nx20: 0000000000000001 x21: 000000003b3d4f78\r\nx22: 0000000000000000 x23: 000000003b3d5ad8\r\nx24: 000000003b3e4e84 x25: 0000000000000000\r\nx26: 0000000000010000 x27: 000000003b3cc000\r\nx28: 000000003af53600 x29: 000000003af3b220\r\n\r\nCode: 17fffff1 f94017e0 79400000 12003c00 (d5033fbf) \r\nResetting CPU …\r\n\r\nresetting …\r\n\r\n\r\n

I’ve updated the firmware on the CM4s (didn’t solve it), and tried Ubuntu 23.04 to see if it had the same issue (nope). I already tried a snapshot of NixOS 23.05 (didn’t solve it either).

Since I want to do some ML on this cluster anyway, I’ve decided to buy a two Coral miniPCI and a Jetson Orin as replacement for slot #3 and call it a day. This device is properly supported on NixOS and Turing Pi 2. Although the guide isn’t for NixOS, the former should be adequate.

Jetpack-nixos - NixOS module for NVIDIA Jetson devices - Announcements - NixOS Discourse