Raspberry Pi NixOS on BTRFS Root?

Learn
#1

Hi all,

I’ve been struggling for a few weeks to get NixOS 21.05 running on a Raspberry Pi 3 on a BTRFS root.

I’ve detailed my process at r/NixOS but thought I would also ask here.

I’ve converted other Pis to BTRFS, which usually entails making a BTRFS partition with the necessary structure, then modifying the boot partition with something like root=UUID=ABCD rootflags=subvol=@root.

So far, I’ve been able to copy the root partition from the NixOS sd_image to a USB key, convert it to BTRFS, and boot to it (using SD card for boot partition, USB as root), but when I try to clone the USB image back to the SD card’s root partition, I get boot-time errors about “unknown filesystem”.

Does anyone have any ideas how I could modify the boot partition to recognize BTRFS? Or perhaps if that’s not the problem at all?

Many thanks in advance!

#2

I wonder if I can somehow customize the uboot with the extraConfig parameter and add CONFIG_FS_BTRFS=y?

#3

Maybe you need something like this: uboot nix derivation for Rock64 with CONFIG_BAUDRATE=115200 · GitHub.

#4

Well, after a lot of googling, I put the following into my configuration.nix:

nixpkgs.overlays = [ (self: super: {
  uboot = super.uboot.override { extraConfig = "CONFIG_FS_BTRFS=y"; };
}) ];

I was able to nixos-rebuild switch, which seemed really encouraging, but after a reboot I found that the u-boot-rpi3.bin is identical to the one from the original SD image (md5sum matches), so apparently that didn’t do anything. (EDIT: bootcode.bin is also unchanged.)

For my next attempt, I put this into btrfs_sd_image.nix and ran the command that is commented out at the top, with the plan to mount the resulting image and see if the u-boot-rpi3.bin has been modified or not.

# nix-build '<nixpkgs/nixos>' -I nixos-config=./btrfs_sd_image.nix -A config.system.build.sdImage
{ pkgs, ... }:
let uboot = pkgs.uboot.override { extraConfig = "CONFIG_FS_BTRFS=y"; };
in {
    imports = [
        <nixpkgs/nixos/modules/installer/sd-card/sd-image-aarch64.nix>
    ];
}
#5

Yeah, that didn’t change anything. The entire FIRMWARE partition is identical to the original. What am I doing wrong here?

#6

Today tried adding boot.initrd.availableKernelModules = ["btrfs"]; to the btrfs_sd_image.nix and again found that the resulting FIRMWARE partition is identical to the default SD image – it’s having no effect at all.

Does anyone know why these changes aren’t affecting the sd image’s boot partition? What am I missing here?

#7

Also tried adding fileSystems = lib.mkForce {} with a few BTRFS subvolumes – nothing, it still ends up with a single ext4 root partition. ¯\(ツ)/¯

#8

I’ve been watching this thread since I had been considering an RPI4. Maybe share the entire hardware.nix and configuration.nix?

#9

When it comes to trying to build the SD image, there’s not much that I’ve left out (other than the specific BTRFS subvolumes I like):

# nix-build '<nixpkgs/nixos>' -I nixos-config=./btrfs_sd_image.nix -A config.system.build.sdImage
{ lib, pkgs, ... }:
{
	imports = [
		<nixpkgs/nixos/modules/installer/sd-card/sd-image-aarch64.nix>
	];
	boot.initrd.availableKernelModules = [
		"btrfs"
	];

# fileSystems = lib.mkForce {
# 	"/boot" = {
# 		device = "/dev/disk/by-label/FIRMWARE";
# 		fsType = "vfat";
# 	};
# 	"/" = {
# 		device = "/dev/disk/by-label/NIXOS_SD";
# 		fsType = "ext4";
# 	};
# };

	fileSystems = lib.mkForce {
		"/" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@" ];
		};
		"/boot" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@boot" ];
		};
		"/var" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@var" ];
		};
		"/home" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@home" ];
		};
		"/nix" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@nix" ];
		};
		"/swap" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "autodefrag" "subvol=@swap" ];
		};
		"/.snapshots" = {
			device = "/dev/disk/by-label/NIXOS_SD";
			fsType = "btrfs";
			options = [ "noatime" "ssd_spread" "discard=async" "compress-force=zstd" "subvol=@snapshots" ];
		};

	};

	swapDevices = [ { device = "/swap/swapfile"; size = 1024; } ];
}
#10

Newest attempt with GitHub - Robertof/nixos-docker-sd-image-builder: Build custom SD images of NixOS for your Raspberry Pi (or any other supported AArch64 device) in 5-20 minutes. – also no luck.