Raspberry Pi NixOS on BTRFS Root?

Hi all,

I’ve been struggling for a few weeks to get NixOS 21.05 running on a Raspberry Pi 3 on a BTRFS root.

I’ve detailed my process at r/NixOS but thought I would also ask here.

I’ve converted other Pis to BTRFS, which usually entails making a BTRFS partition with the necessary structure, then modifying the boot partition with something like root=UUID=ABCD rootflags=subvol=@root.

So far, I’ve been able to copy the root partition from the NixOS sd_image to a USB key, convert it to BTRFS, and boot to it (using SD card for boot partition, USB as root), but when I try to clone the USB image back to the SD card’s root partition, I get boot-time errors about “unknown filesystem”.

Does anyone have any ideas how I could modify the boot partition to recognize BTRFS? Or perhaps if that’s not the problem at all?

Many thanks in advance!

I wonder if I can somehow customize the uboot with the extraConfig parameter and add CONFIG_FS_BTRFS=y?

Maybe you need something like this: uboot nix derivation for Rock64 with CONFIG_BAUDRATE=115200 · GitHub.

Well, after a lot of googling, I put the following into my configuration.nix:

nixpkgs.overlays = [ (self: super: {
  uboot = super.uboot.override { extraConfig = "CONFIG_FS_BTRFS=y"; };
}) ];

I was able to nixos-rebuild switch, which seemed really encouraging, but after a reboot I found that the u-boot-rpi3.bin is identical to the one from the original SD image (md5sum matches), so apparently that didn’t do anything. (EDIT: bootcode.bin is also unchanged.)

For my next attempt, I put this into btrfs_sd_image.nix and ran the command that is commented out at the top, with the plan to mount the resulting image and see if the u-boot-rpi3.bin has been modified or not.

# nix-build '<nixpkgs/nixos>' -I nixos-config=./btrfs_sd_image.nix -A config.system.build.sdImage
{ pkgs, ... }:
let uboot = pkgs.uboot.override { extraConfig = "CONFIG_FS_BTRFS=y"; };
in {
    imports = [
        <nixpkgs/nixos/modules/installer/sd-card/sd-image-aarch64.nix>
    ];
}

Yeah, that didn’t change anything. The entire FIRMWARE partition is identical to the original. What am I doing wrong here?

Today tried adding boot.initrd.availableKernelModules = ["btrfs"]; to the btrfs_sd_image.nix and again found that the resulting FIRMWARE partition is identical to the default SD image – it’s having no effect at all.

Does anyone know why these changes aren’t affecting the sd image’s boot partition? What am I missing here?

Also tried adding fileSystems = lib.mkForce {} with a few BTRFS subvolumes – nothing, it still ends up with a single ext4 root partition. ¯\(ツ)

I’ve been watching this thread since I had been considering an RPI4. Maybe share the entire hardware.nix and configuration.nix?

When it comes to trying to build the SD image, there’s not much that I’ve left out (other than the specific BTRFS subvolumes I like):

# nix-build '<nixpkgs/nixos>' -I nixos-config=./btrfs_sd_image.nix -A config.system.build.sdImage
{ lib, pkgs, ... }:
{
	imports = [
		<nixpkgs/nixos/modules/installer/sd-card/sd-image-aarch64.nix>
	];
	boot.initrd.availableKernelModules = [
		"btrfs"
	];

# fileSystems = lib.mkForce {
# 	"/boot" = {
# 		device = "/dev/disk/by-label/FIRMWARE";
# 		fsType = "vfat";
# 	};
# 	"/" = {
# 		device = "/dev/disk/by-label/NIXOS_SD";
# 		fsType = "ext4";
# 	};
# };

	fileSystems = lib.mkForce {
		"/" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@" ];
		};
		"/boot" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@boot" ];
		};
		"/var" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@var" ];
		};
		"/home" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@home" ];
		};
		"/nix" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@nix" ];
		};
		"/swap" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@swap" ];
		};
		"/.snapshots" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "subvol=@snapshots" ];
		};

	};

	swapDevices = [ { device = "/swap/swapfile"; size = 1024; } ];
}

Newest attempt with GitHub - Robertof/nixos-docker-sd-image-builder: Build custom SD images of NixOS for your Raspberry Pi (or any other supported AArch64 device) in 5-20 minutes. – also no luck.