Btrfs raid with Disko

Help
1

First time installing NixOS manually and fully declaratively. Also first time using disko, so i have a few questions i could not find answers for:

  1. if i determine already existing, exactly same partition in Disko, will it reformat disk?
  2. i have a btrfs raid between 2 SSDs (not mdadm), how can i determine this structure in disko configuration?
3 Likes
2

To 1

If you have the following configuration in your flake:

# configuration.nix / flake.nix
imports = with inputs; [
  ./disk-configuration.nix
  disko.nixosModules.disko
];

(With my current limited knowledge:) No, it will only reformat the disk with the command disko --mode destroy,format,mount disk-configuration and when using the nixos-anywhere installer. And it will ask you, before deleting data (exept with --yes-wipe-all-disks). Still… make sure you have a backup of your data before running any disko-command!

But it will try to mount your newly declared disks. Disko is a wrapper of the nixos-option fileSystems. So make sure to delete the duplicate e.g. fileSystems."/" or fileSystems."/boot" depending on your disk layout in your hardware-configuration.nix. You can check, what disko does (before running the command nixos-rebuild --flake .# boot) with nixos-rebuild --flake .# repl and then > config.fileSystems.

To 2

I have done it the following way:

# disk-configuration.nix
disko.devices.disk = {
  "myBtrfsRaid1" = {
    type = "disk";
    device = "/dev/disk/by-id/<...disk1...>";
    content = {
      type = "gpt";        # TIP: omit this line for a btrfs-rawdisk
      partitions = {       # TIP: omit this line for a btrfs-rawdisk
        "main" = {         # TIP: omit this line for a btrfs-rawdisk
          size = "100%";   # TIP: omit this line for a btrfs-rawdisk
          content = {      # TIP: omit this line for a btrfs-rawdisk
            type = "btrfs";
            extraArgs = [
              "--force"
              "--data raid1"
              "--metadata raid1"
              "/dev/disk/by-id/<...disk2...>"
              #"/dev/disk/by-id/<...disk3...>"
              # with 4 disks you can use `raid10`
            ];
            subvolumes = {
              "mySubvolume" = {
                mountpoint = "/mnt/mySubvolume";
                mountOptions = [
                  "compress=zstd"
                  "defaults"
                  "x-gvfs-show"
                ];
              };
            };
          };               # TIP: omit this line for a btrfs-rawdisk
        };                 # TIP: omit this line for a btrfs-rawdisk
      };                   # TIP: omit this line for a btrfs-rawdisk
    };
  };
};

This will run the following formatting command for btrfs as you can see in the disko-implementation:

mkfs.btrfs '/dev/disk/by-id/<...disk1...>' --force --data raid1 --metadata raid1 '/dev/disk/by-id/<...disk2...>'

The drawback i can think of: When mounting, disko will only look on disk1 for your btrfs filesystem, so i dont know what happens, when disk1 fails. I think you will have to put the other disk into the device-option, if disk1 fails.

2 Likes
3

I wrote a helper function mkBtrfsRAID to make defining btrfs raid collections more ergonomic in disko. It has the “other” disks formatted with an empty partition table, so disko knows about them and asks for wipe confirmation on formatting. Doesn’t solve the abovementioned problem when the first disk that’s actually tried to be mounted fails or is missing though.

let
  # make a disko.devices.disk attrset that empty-format the given disks before anything else
  mkEmptyDisks = { before ? "", devices }:
    builtins.listToAttrs (builtins.genList (i:
      let dev = builtins.elemAt devices i;
      in {
        # disks are formatted alphabetically, these dummys must be formatted first, so we prefix with zeros
        name = "#before:${before}:${builtins.toString i}";
        value = {
          type = "disk";
          device = dev;
          content = {
            type = "gpt";
            partitions = { };
          };
        };
      }) (builtins.length devices));
  # make a disko.devices.disk attrset forming a btrfs raid from a list of devices
  # The *first* device in this list will be mounted (which will auto-mount the others)
  mkBtrfsRAID = { name, devices, raid ? "raid1", content ? { } }:
    {
      "${name}" = {
        type = "disk";
        # The first device in the list is used, but it could be any of those
        device = builtins.head devices;
        content = content // {
          type = "btrfs";
          extraArgs = (content.extraArgs or [ ])
            ++ [ "-f" "-d ${raid}" "-m ${raid}" ] ++
            # disko has no builtin concept of collections of btrfs disks,
            # so we sneak all the other parts of the RAID into the creation of the "first" one
            # Below, all of these other disks are first formatted empty
            # to ensure disko knows about them and asks the user for confirmation upon formatting
            builtins.tail devices;
        };
      };
    }
    # This will first empty-format all the other parts of the RAID
    # This ensures disko will ask for confirmation to empty those out
    // mkEmptyDisks {
      before = name;
      devices = builtins.tail devices;
    };
in {
  disko.devices = {
    disk = mkBtrfsRAID {
      name = "HDD-raid";
      raid = "raid1";
      devices = [
        # ⚠️  the *first* disk here will be mounted (which will auto-mount the others)
        "/dev/disk/by-id/ata-..._1234"
        "/dev/disk/by-id/ata-..._2345"
        "/dev/disk/by-id/ata-..._3456"
        "/dev/disk/by-id/ata-..._4567"
      ];
      content = {
        # mount the entire root subvolume for easy snapshotting "across" subvolumes
        mountpoint = "/mnt/.DATA";
        mountOptions = [ "compress=zstd" "noatime" ];
        subvolumes = {
          "services" = {
            mountpoint = "/var/lib";
            mountOptions = [ "compress=zstd" "noatime" ];
          };
          "data" = {
            mountpoint = "/data";
            mountOptions = [ "compress=zstd" "noatime" ];
          };
        };
      };
    };
  };
}
2 Likes