[HELP] Nixos configuration for UEFI - ZFS - luksEncrypted

Hi,

I’ve been a NixOS User for at least 4-5 months and I’ve recently started a little (homelab) project which requires installing nixos with

  • ZFS
  • Full Disk encryption
  • UEFI Boot

I think I am almost done. Grub it’s working, password for decrypting the disk it’s working, systemd starts… Ok (few errors which I’m trying to recover through journalctl) and then I find myself in an empty terminal, which tells me I’m in systemd emergency mode.

I am pretty sure that the error in my configuration it’s in these few lines of code:

  boot = {
    supportedFilesystems = [ "zfs" ];
    loader = {
      systemd-boot.enable = true;
      grub = {
        enable = true;
        version = 2;
        device = "nodev";
        efiSupport = true;
        enableCryptodisk = true;
      };
      efi = {
        canTouchEfiVariables = true;
        efiSysMountPoint = "/boot";
      };
    };
    kernelParams = [ "nohibernate" ];
    initrd = {
      supportedFilesystems = [ "zfs" ];
      luks.devices = {
        root = {
          device = "/dev/disk/by-uuid/91e9b387-f945-4466-8fa0-2a9bce81fa97";
          preLVM = true;
        };
      };
    };
  };

Could you help me out with this problem? thanks a lot

N.B.

  • The rest of the configuration.nix it’s correct, since it’s what I’ve used with close-to-0 modifications on my main PC ant several VMs. So the errors are there.

In case you find a working Uefi-zfs-luks compatible nixos config, that would help a lot.


I managed to figure out something more about my current problem… so, I have /dev/sda3 luks encrypted → /dev/mapper/cryptroot
then, I created a zfs pool on cryptroot → rpool
and then i did

zfs create -o mountpoint=legacy rpool/root
zfs create -o mountpoint=legacy rpool/root/nixos
zfs create -o mountpoint=legacy rpool/home
zfs create -o mountpoint=legacy rpool/var

(all that after creating their respective directories)

Problems

Those are all the errors I could get during the boot phase

[FAILED] Failed to start Remount Root and Kernel File Systems.
See 'systemctl status systemd-remount-fs.service' for details
systemd [1] failed to mount /home
[FAILED] Failed to mount /home
See 'systemctl status home.mount' for details
[DEPEND] Dependency failed for Local File Systems.

Example of my hardware-config.nix

{ config, lib, pkgs, modulesPath, ... }:

{
  imports =
    [ (modulesPath + "/installer/scan/not-detected.nix")
    ];

  boot.initrd.availableKernelModules = [ "xhci_pci" "ahci" "nvme" "usb_storage" "usbhid" "sd_mod" ];
  boot.initrd.kernelModules = [ ];
  boot.kernelModules = [ "kvm-amd" ];
  boot.extraModulePackages = [ ];

  fileSystems."/" =
    { device = "rpool/root/nixos";
      fsType = "zfs";
    };

  fileSystems."/home" =
    { device = "rpool/home";
      fsType = "zfs";
      options = [ "zfsutil" ];
    };

  fileSystems."/var" =
    { device = "rpool/var";
      fsType = "zfs";
      options = [ "zfsutil" ];
    };

  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/C632-2B34";
      fsType = "vfat";
    };

  swapDevices =
    [ { device = "/dev/disk/by-uuid/4110c69c-c13b-442e-94a7-57c46667dc6e"; }
    ];

  hardware.cpu.amd.updateMicrocode = lib.mkDefault config.hardware.enableRedistributableFirmware;
}

Systemctl status home.mount

Loaded: /etc/fstab (generated)
Active: failed (Result: exit-code)
Where: /home
What: rpool/home

filesystem 'rpool/home' cannot be mounted using zfs-mount
use 'zfs set mountpoint=/home' or 'mount -t zfs rpool/home /home'
see zfs(8) for more information.

Systemctl status system-remount-fs

Loaded: loaded (/etc/systemd/system/system-remount-fs.service; enabled-runtime; vendor preset: enabled)
Drop-In: /nix/store/44789...1nsg-system-units/systemd-remount-fs.service.d
         |-overrides.conf
Active: failed (Result: exit-code)

filesystem 'rpool/root/nixos' cannot be mounted using zfs-mount
use 'zfs set mountpoint=/' or 'mount -t zfs rpool/root/nixos /'
see zfs(8) for more information.

hi, I used this setup, that worked fine, I only changed the partitioning a little ( esp as first and zfs as second)

https://setkeh.com/posts/nixos-zfs-install-guide/

maybe you can compare it with your setup method

Unfortunately, that’s exactly the guide I followed… I’ll try another time, just to be sure to have followed all the steps.

in my configuration, I did not add grub, I just used the generated version, so maybe first use the simplest configuration possible, later when it boots, adapt it step by step?

here the first part of mine (/dev/sda1 is esp, /dev/sda2 is luks zfs):

{ config, pkgs, ... }:

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
    ];
  nixpkgs.config.allowUnfree = true; 

  # Use the systemd-boot EFI boot loader.
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  boot.supportedFilesystems = [ "zfs" ];

  boot.initrd.luks.devices = {
   root = {
     device = "/dev/sda2"; 
     preLVM = true;
   };
  };

  networking.hostName = "nixos"; # Define your hostname.
  networking.hostId = "deadbeef"; # change some digits!

  # networking.wireless.enable = true;  # Enables wireless support via wpa_supplicant.
  networking.networkmanager.enable = true;

  # Set your time zone.
  time.timeZone = "Europe/Amsterdam";

so these are the most important changes from the default generated configuration.nix:

# now add to configuration.nix

boot.supportedFilesystems = [ "zfs" ];

networking.hostId = "deadbeef"; # change some 8 hex digits, zfs needs this

boot.initrd.luks.devices = {
   root = {
     device = "/dev/sda2
     preLVM = true;
   };
  };

After re-following the guide correctly, now everything works. I think it was due to the useless option that I added at first. I followed the guide, keeping the most minimal, but working config, just some options about boot, network, timezone.

Changes

  boot.supportedFilesystems = [ "zfs" ];

  boot.loader.efi.canTouchEfiVariables = true;

  networking.hostId = "00000098";
  boot.loader.systemd-boot.enable = true;

  boot.loader.grub = {
    enable = true;
    version = 2;
    device = "nodev";
    efiSupport = true;
    enableCryptodisk = true;
  };
  boot.loader.efi.efiSysMountPoint = "/boot";
  boot.initrd.luks.devices = {
   root = {
     device = "/dev/disk/by-uuid/93n49cm....";
     preLVM = true;
   };
  };

P.S. All the guides I found used GRUB, but since nixos ships with systemd by default, i think I’ll continue to use the default option.

P.S.2. Can confirm that this guide works.

IMPORTANT

Also, I discovered that in hardware-configuration.conf this option

options = [ "zfsutil" ];

(in my case) it’s exactly what causes the systemd boot problem (emergency mode) since it seems that during the boot, with this option loaded, the system tries to mount the “zfsutil datasets” to the directiories in a wrong way

Any clue how to change the grub boot into systemd - boot ?

Took me so long to set ZFS UEFI lukscrypt and your guide finally saved me.

All the documentation configs such as

services.udev.extraRules = ''
  ACTION=="add|change", KERNEL=="sd[a-z]*[0-9]*|mmcblk[0-9]*p[0-9]*|nvme[0-9]*n[0-9]*p[0-9]*", ENV{ID_FS_TYPE}=="zfs_member", ATTR{../queue/scheduler}="none"

were not needed at all.