Problem booting after 21.11 -> 22.05 upgrade (FDE)

Hi everyone, first time posting on this forum!

I recently upgraded from 21.11 to 22.05 on my NixOS machine, and I am now unable to boot without manual intervention my machine.

For some context, my machine is using full disk encryption, and has two different disks.

The setup is the following:

  • Disk 1
    • /boot
      • LUKS (MainGroup)
        • /
  • Disk 2
    • LUKS (ExtraGroup)
      • /extra
      • swap

Everything worked correctly on 21.11, but since switching to 22.05, I need to drop to the emergency shell and run vgchange -ay for the partitions in ExtraGroup to be detected correctly.

I sadly did not find anything online about a similar issue in NixOS.

Hoping someone can help me shed some light on this!

I tested this in a VM and it boots just fine on 22.05 for me :confused: After installing NixOS and booting, lsblk looks like this:

[root@nixos:~]# lsblk
NAME                   MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINTS
fd0                      2:0    1    4K  0 disk  
sda                      8:0    0   10G  0 disk  
├─sda1                   8:1    0    1G  0 part  /boot
└─sda2                   8:2    0    9G  0 part  
  └─MainLUKS           254:1    0    9G  0 crypt 
    └─MainGroup-MainLV 254:2    0    9G  0 lvm   /nix/store
                                                 /
sdb                      8:16   0   40G  0 disk  
└─ExtraLUKS            254:0    0   40G  0 crypt 
  └─ExtraGroup-ExtraLV 254:3    0   40G  0 lvm   /extra
sr0                     11:0    1 1024M  0 rom   

Configuration.nix:

{ pkgs, ... }: {
  imports = [./hardware-configuration.nix];

  boot.initrd.luks.devices = {
    MainLUKS.device = "/dev/disk/by-uuid/25bf3207-3b52-4b1f-833f-69e39b362e3d";
    ExtraLUKS.device = "/dev/disk/by-uuid/b7285b7b-8bc5-4eaf-9673-988415b99e59";
  };

  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = false;

  services.getty.autologinUser = "root";
}

Hardware-configuration.nix:

# Do not modify this file!  It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations.  Please make changes
# to /etc/nixos/configuration.nix instead.
{ config, lib, pkgs, modulesPath, ... }:

{
  imports =
    [ (modulesPath + "/profiles/qemu-guest.nix")
    ];

  boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd" "virtio_pci" "floppy" "sd_mod" "sr_mod" "virtio_blk" ];
  boot.initrd.kernelModules = [ "dm-snapshot" ];
  boot.kernelModules = [ "kvm-amd" ];
  boot.extraModulePackages = [ ];

  fileSystems."/" =
    { device = "/dev/disk/by-uuid/3e5c3483-1db6-4565-9b59-e2c6da498abe";
      fsType = "ext4";
    };

  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/73A6-94F9";
      fsType = "vfat";
    };

  fileSystems."/extra" =
    { device = "/dev/disk/by-uuid/339bc3da-cec0-4c76-9175-02c5d6babe61";
      fsType = "ext4";
    };

  swapDevices = [ ];

  # Enables DHCP on each ethernet and wireless interface. In case of scripted networking
  # (the default) this is the recommended approach. When using systemd-networkd it's
  # still possible to use this option, but it's recommended to use it in conjunction
  # with explicit per-interface declarations with `networking.interfaces.<interface>.useDHCP`.
  networking.useDHCP = lib.mkDefault true;
  # networking.interfaces.eth0.useDHCP = lib.mkDefault true;

  hardware.cpu.amd.updateMicrocode = lib.mkDefault config.hardware.enableRedistributableFirmware;
}

Kind of annoying that nixos-generate-config didn’t generate the LUKS expressions for me, but oh well.

I adapted your config to my situation and it works perfectly, thank you so much!

My config was the following:

{
    boot.initrd = {
      luks.devices = {
        root = {
          device = "/dev/disk/by-uuid/[uuid main group]";
          allowDiscards = true;
          keyFile = "/keyfile_main.bin";
          preLVM = true;
        };
      };
      secrets = {
        "keyfile_main.bin" = "/etc/secrets/initrd/keyfile_main.bin";
        "keyfile_extra.bin" = "/etc/secrets/initrd/keyfile_extra.bin";
      };
    };
  fileSystems."extra" = {
    device = "/dev/disk/by-uuid/[uuid extra part]";
    mountPoint = "/extra";
    encrypted = {
      enable = true;
      label = "ExtraGroup";
      blkDev = "/dev/disk/by-uuid/[uuid extra group]";
      keyFile = "/keyfile_extra.bin";
    };
  };
 fileSystems."/" =
    { device = "/dev/disk/by-uuid/[uuid root part]";
      fsType = "ext4";
    };

  fileSystems."/boot/efi" =
    { device = "/dev/disk/by-uuid/[uuid boot part]";
      fsType = "vfat";
    };
}

I guess something about the non-boot encrypted volumes has changed between 21.11 and 22.05.

Your old config looks like you had it configured for the LV being encrypted instead of the physical disk, except you also had preLVM = true;. I cannot imagine how that ever worked :stuck_out_tongue:

2 Likes

I don’t really know either, but it worked for a few years ^^

Thanks again for your time!