Zfs encrypted boot installation: no password prompt

Help
1

Cross-posting from Stack Overflow: Nixos zfs encrypted boot installation - Stack Overflow

I’ve followed several guides[1,2,3] but cannot get this working. Grub does not prompt me for my password to unlock the disk, and instead I am greeted with this prompt:

error: no such device: 397411472d225490.
error: unknown filesystem.
Entering rescue mode...
grub rescue>

These are my configuration file:

/etc/nixos/configuration.nix:

{ config, pkgs, ... }:

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
    ];

  boot.supportedFilesystems = [ "zfs" ];

  boot.initrd.luks.devices."nixos-root" = {
    device = "/dev/disk/by-uuid/f7b6b840-d3c3-4906-ae32-e33b43dd9c76";
    keyFile = "/root/root.key";
    allowDiscards = true;
  };

  boot.loader = {
    efi = {
      efiSysMountPoint = "/efi";
      canTouchEfiVariables = true;
    };

    grub = {
      enable = true; 
      device = "nodev";
      version = 2;
      efiSupport = true;
      enableCryptodisk = true;
      extraInitrd = "/boot/initrd.keys.gz";
      zfsSupport = true;
    };

  };

  networking.hostName = "lostpuppy";
  networking.hostId = "0261fc86";

...
}

/etc/nixos/hardware-configuration.nix:

{ config, lib, pkgs, ... }:

{
  imports =
    [ <nixpkgs/nixos/modules/installer/scan/not-detected.nix>
    ];

  boot.initrd.availableKernelModules = [ "xhci_pci" "nvme" "rtsx_usb_sdmmc" ];
  boot.initrd.kernelModules = [ ];
  boot.kernelModules = [ "kvm-intel" ];
  boot.extraModulePackages = [ ];

  fileSystems."/" =
    { device = "zroot/root/nixos";
      fsType = "zfs";
    };

  fileSystems."/home" =
    { device = "zroot/home";
      fsType = "zfs";
    };

  fileSystems."/efi" =
    { device = "/dev/disk/by-uuid/E7FC-1575";
      fsType = "vfat";
    };

  nix.maxJobs = lib.mkDefault 8;
  powerManagement.cpuFreqGovernor = lib.mkDefault "powersave";
}

I hope someone can help me!

[1] https://zyradyl.moe/2019/08/04/NixOS-Part1-Basic-Setup/ (currently offline)

[2] Encrypted /boot on ZFS with NixOS

[3] NixOS install with encrypted /boot /root with single password unlock · GitHub

1 Like
2

I got encrypted ZFS working on my laptop just the other day, primarily with the help from the Wiki.

https://nixos.wiki/wiki/NixOS_on_ZFS#Encrypted_ZFS

The important parts from my configuration.nix…

# Use the systemd-boot EFI boot loader.
boot.loader.systemd-boot.enable = true;
boot.loader.efi.canTouchEfiVariables = true;

# Add ZFS support.
boot.supportedFilesystems = [ "zfs" ];
boot.zfs.requestEncryptionCredentials = true;
networking.hostId = "78e2f8a1";
services.zfs.autoSnapshot.enable = true;
services.zfs.autoScrub.enable = true;

My hardware-configuration.nix

{ config, lib, pkgs, ... }:

{
  imports =
    [ <nixpkgs/nixos/modules/installer/scan/not-detected.nix>
    ];

  boot.initrd.availableKernelModules = [ "xhci_pci" "ahci" "usb_storage" "sd_mod" "rtsx_pci_sdmmc" ];
  boot.initrd.kernelModules = [ ];
  boot.kernelModules = [ "kvm-intel" ];
  boot.extraModulePackages = [ ];

  fileSystems."/" =
    { device = "zroot/root/nixos";
      fsType = "zfs";
    };

  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/4D94-FF66";
      fsType = "vfat";
    };

  fileSystems."/tmp" =
    { device = "zroot/root/tmp";
      fsType = "zfs";
    };

  fileSystems."/home" =
    { device = "zhome/root/home";
      fsType = "zfs";
    };

  swapDevices =
    [ { device = "/dev/disk/by-uuid/cf41644a-5259-4bf4-be95-1d923edc5c63"; }
    ];

  nix.maxJobs = lib.mkDefault 4;
  powerManagement.cpuFreqGovernor = lib.mkDefault "powersave";
}

I have two disks, first has /boot for EFI, and zroot zpool for / and /tmp, second has zhome zpool for /home.

Both zpools are encrypted, with zroot prompting for passphrase on boot, and zhome using key file stored on /.

It took a few tries to get it working, especially as the generated hardware-configuration.nix had entries for /zroot and /zhome mounts that seem to throw errors. Had to comment them out.

Hope that helps.

3

Grub’s enableCryptodisk doesn’t work with LUKS2, unfortunately. Only LUKS1.

@ianmjones I’m assuming your /boot isn’t encrypted? Pretty sure neither grub nor systemd-boot support reading encrypted ZFS datasets to find the kernel and initrd.

4

Correct, /boot isn’t encrypted.

5

On my installation /boot is supposed to be encrypted, and lives on the root partition, if I understand correctly. Is there a way to know if I am using LUKS1 or LUKS2? Thanks.

6

@ElvishJerricco I just saw what you wrote here:

UPDATE: This does not work with LUKS 2, because Grub does not support reading from it. At least on NixOS, LUKS 2 is now the default. I have not updated this post to show how to force it to use LUKS 1, and I’m not sure that’s a good idea anyway. I’m moving my system back to a normal /boot because of this and because Grub takes absolutely forever to decrypt the disk. Grub also has issues with new ZFS feature flags on occasion, and I’m not willing to take that risk.

So I guess I am doomed? Do I need to start all over from scratch, or is there an easy way to change to a normal /boot at this point?

7

If that /efi partition is big enough, you could repurpose it as a /boot partition and have the efi directory be on that partition at /boot/efi. Dunno if you could just downgrade to LUKS1, or what the risks of that would be.

8

Thanks. How big is “big enough”? It’s 200M. I read the 512M is typical?

9

My /boot has a /boot/EFI/, is 512M in size, but only 49M is currently in use. So you may get away with 200M.

You might just have to be extra careful about pruning older generations to make sure you don’t fill it up.

But I’m not very clued up in this area, there may be other factors that make having a 200M /boot a bit of problem.

10

Aldo, i use boot.zfs.enableUnstable = true;

11

200M is common, but 512M is definitely better. People who keep lots of old NixOS generations around tend to fill up 200M with a bunch of excess kernels and stuff.

12

이 문제 해결되었습니까?

13

@ ianmjones

How did you set the keyfile for zhome? This step isn’t really spelled out in the wiki.

Thanks!

14

I wish I’d taken notes of how I finally got it working, it was while ago now!

I believe that the following statement in the wiki article was key to my success…

Note that at the moment one can only use passphrases ( keylocation=prompt ) for pools that are mounted as the root fs.

So Ideally when you create the second encrypted zhome pool with zpool, you should specify keylocation as a file that is created at the root of the zroot pool that does use the passphrase prompt on boot.

If I remember correctly, I did it wrong at first, using keylocation=prompt, but later fixed it because zroot was ok, I just got messages that zhome couldn’t be mounted. I then created my /zhome.key file, manually mounted the zhome pool and used zfs change-key ... to fix it up, once I’d read that above phrase a million times!

So, maybe something like this is better after creating and mounting zroot on /mnt and creating the key file?

zpool create -o ashift=12 -o altroot="/mnt/home" -O mountpoint=none -O encryption=aes-256-gcm -O keyformat=passphrase -O keylocation=/zhome.key zhome /dev/disk-by-uuid/SOMETHING

My zhome.key file in / (a.k.a. /mnt/zhome.key during setup), is just a plain file that simply contains the passphrase (password) that I was asked for when I initially screwed things up.

Hope that helps.