LUKS-encrypted LVM disk not mounting on boot

Help
1

Hello,

I’m having an issue with mounting a LUKS-encrypted LVM disk on boot. Here’s what I’ve done:

  1. I have a spare SSD (/dev/nvme1n1) that I wanted to encrypt and mount in NixOS.
  2. I set up LVM and LUKS encryption on the disk:
pvcreate /dev/nvme1n1
vgcreate vg_crypt /dev/nvme1n1
lvcreate -l 100%FREE -n lv_crypt vg_crypt
cryptsetup luksFormat /dev/vg_crypt/lv_crypt
cryptsetup luksOpen /dev/vg_crypt/lv_crypt crypt_device
mkfs.ext4 /dev/mapper/crypt_device
mount /dev/mapper/crypt_device /home/wanten/disk1
  1. The lsblk -f output shows:
NAME     FSTYPE    FSVER    LABEL UUID    FSAVAIL FSUSE% MOUNTPOINTS
nvme1n1   LVM2_member LVM2 001  WAaFB4-ljOF-AKRH-L8Rb-ZQnF-Gf11-aLpknj
└─vg_crypt-lv_crypt   crypto_LUKS 2    2b723313-e459-4a53-b209-cd82c8274fb1
└─crypt_device    ext4     1.0   919180ad-fe21-43a0-a64a-70031a8b4d1d    444.5G     0% /home/wanten/disk1
  1. I added the following to my hardware.nix configuration:
...
boot.initrd.availableKernelModules = [ "xhci_pci" "ahci" "nvme" "usbhid" "usb_storage" "sd_mod" ];
  boot.initrd.kernelModules = [ "dm-snapshot" ];
  boot.kernelModules = [ "kvm-intel" ];
  boot.extraModulePackages = [ ];

fileSystems."/home/wanten/disk1" =
  {
    device = "/dev/disk/by-uuid/919180ad-fe21-43a0-a64a-70031a8b4d1d";
    fsType = "ext4";
  };
boot.initrd.luks.devices."crypt_disk1".device = "/dev/disk/by-uuid/2b723313-e459-4a53-b209-cd82c8274fb1";
...
  1. nixos-rebuild switch was successful.
    However, when I reboot, I get stuck at the NixOS Stage 1 with this error:
Waiting 10 seconds for device /dev/disks/by-uuid/2b723313-e459-4a53-b209-cd82c8274fb1 to appear.......
An error occured in stage 1 of the boot process, which must mount the
root filesystem on `/mnt-root` and then start stage 2. Press one
of the following keys:
  r) to reboot immediately
  *) to ignore the error and continue

It seems that the system can’t find or mount the encrypted disk during boot. What am I missing in my configuration? How can I properly set up NixOS to mount this LUKS-encrypted LVM disk on boot?

Thank you for your help!

2

Try

boot.initrd.luks.devices."crypt_disk1".device = /dev/vg_crypt/lv_crypt";
3

Thank you for your quick response and suggestion!

I had actually already tried the method you suggested. Unfortunately, it didn’t resolve the issue in my case. The system still fails to find the encrypted disk during boot.

Waiting 10 seconds for device /dev/vg_crypt/lv_crypt .....
4

Long shot here, in case you are flexible in this regard; I’ve been using encrypted ZFS and never had difficulty setting up boot up passphrase prompt. It kind of just works. If you like, I can share my related config.

5

Thank you for offering an alternative. Are these the two configurations you mentioned? I might give them a try:

6

Yup. Good luck. I find ZFS easier, because it’s a single thingo for volume management, encryption and filesystem.

7

I tried ZFS but failed. After entering the ZFS password at the boot screen, it showed “zfs mountpoint or dataset is busy” and entered emergency root mode. Even if I succeeded, I wouldn’t use ZFS because it cannot reuse the LUKS password entered during NixOS Stage1, meaning I would have to enter the password twice to boot successfully.

I’ve decided that if I can’t successfully use a LUKS-encrypted LVM disk, I won’t decrypt and mount the disk at boot. Instead, I will mount it when needed, writing a script to handle this task.

8

Reuse the LUKS password? ZFS has its own encryption. Sorry, I didn’t mean to suggest you use both LUKS and encrypted ZFS.

9

I understand, thanks for clarifying. My main concern is to minimize the number of passwords I need to enter at boot. Since ZFS encryption and LUKS would require separate passwords, it complicates things for my setup.

I’ve managed to achieve what I want with a service:

{ config, pkgs, username, ... }:

{
  systemd.services.mount-mini-disk = {
    description = "Mount encrypted mini-disk";
    wantedBy = [ "multi-user.target" ];
    after = [ "network.target" ];
    path = [ pkgs.cryptsetup ];

    serviceConfig = {
      Type = "oneshot";
      RemainAfterExit = true;
      User = "root";
      ExecStart = pkgs.writeScript "mount-mini-disk.sh" ''
        #!${pkgs.bash}/bin/bash
        ${pkgs.cryptsetup}/bin/cryptsetup luksOpen /dev/disk/by-uuid/e8807be5-9d00-4d3b-9846-81b27811bbee mini-disk --key-file ${config.age.secrets.disk-key.path}
        ${pkgs.coreutils}/bin/mkdir -p /home/${username}/mini-disk
        ${pkgs.utillinux}/bin/mount /dev/mapper/mini-disk /home/${username}/mini-disk
        ${pkgs.coreutils}/bin/chown ${username} /home/${username}/mini-disk
        ${pkgs.coreutils}/bin/chmod 750 /home/${username}/mini-disk
      '';
      ExecStop = pkgs.writeScript "unmount-mini-disk.sh" ''
        #!${pkgs.bash}/bin/bash
        ${pkgs.utillinux}/bin/umount /home/${username}/mini-disk
        ${pkgs.cryptsetup}/bin/cryptsetup luksClose mini-disk
      '';
    };
  };
}
10

What you had originally would have been fine. All you were missing was boot.initrd.luks.devices."crypt_disk1".preLVM = false;. Basically, stage 1 has distinct phases in which it attempts to do things, and by default encrypted devices are found before LVM devices.

This is one thing that the systemd-based initrd will improve upon. boot.initrd.systemd.enable = true; would have enabled this, and it would have handled this without any configuration.

Finally, initrd doesn’t even technically have to be involved at all in this, because the file system isn’t required to reach stage 2. Rather than using boot.initrd.luks.devices.crypt_disk1, you could just configure /etc/crypttab like this:

environment.etc."crypttab".text = ''
  crypt_disk1 /dev/disk/by-uuid/2b723313-e459-4a53-b209-cd82c8274fb1
'';

And this would have done the decryption in stage 2 instead (in exactly the same way that systemd-based initrd would have, just during stage 2 instead of the more limited stage 1 initrd environment)

So, TL;DR: There were sort of three different ways that could have solved the problem. 1) Do it with preLVM = false;, 2) Do it with systemd in stage 1, 3) Do it with systemd in stage 2 via /etc/crypttab.

1 Like
11

Thank you for the detailed explanation.

I hope your methods would work, but I don’t want to try them at this point.

I have already configured my system and believe it offers better security than manually entering a passphrase. I generated a very secure key file and use agenix to automatically decrypt and mount the disk. This way, I don’t even know the password, but I know where the key file is and which other public key it is encrypted with.

12

You can use a key file the other ways as well. Easiest would be the third one. Just add a path to the keyfile in the crypttab entry:

environment.etc."crypttab".text = ''
  crypt_disk1 /dev/disk/by-uuid/2b723313-e459-4a53-b209-cd82c8274fb1 ${config.age.secrets.disk-key.path}
'';
1 Like
13

I can’t believe it was that simple! I tried your suggestion with the following code:

environment.etc."crypttab".text = ''
  mini-disk /dev/disk/by-uuid/e8807be5-9d00-4d3b-9846-81b27811bbee ${config.age.secrets.disk-key.path}
'';
fileSystems."/home/${username}/mini-disk" = {
  device = "/dev/mapper/mini-disk";
  fsType = "ext4";
};

And it worked! Your method is so straightforward! Thank you so much for your help!

1 Like