NixOS LUKS encryption using a YubiKey


I am following the Wiki and about encryption using LUKS and a YubiKey on NixOS.
However, when I plug my YubiKey and wait at startup it tells me that the Authentication failed and:

An error occurred in stage 1 of the boot process, which must mount the root filesystem on `/mnt-root` and then start stage 2.

Do I need to format a certain way or anything else specific?


The error message is too generic to know where the problem lies. If initrd thinks the root disk is /dev/sdb and there is no /dev/sdb, this error would likely present, so the error message doesn’t provide any details specific to where the failure lies. E.g. Is initrd looking for a missing disk? Is initrd able to find the YubiKey? Can it talk to the YubiKey? Did initrd access the correct YubiKey slot? Etc.

Provide a detailed write up of the process you followed with secrets redacted. Then we could compare what you did with the two references you provided and get an idea of any steps you missed. Or sgillespie, or someone else with working LUKS + YubiKey, could better identify where your configuration went wrong if this catches their attention.

At first glance, I figured you might be missing the modules necessary for the initrd to find/talk with the YubiKey. Both references include the same kernel modules.

The config setting for boot.initrd.luks.cryptoModules appears to be absent from the github tutorial. You mention the Wiki [1], which the github tutorial is based on, so I presume you noticed the slight, and compensated, but I’m just guessing in the dark at this point with so little info to work with.


[1]: Yubikey based Full Disk Encryption (FDE) on NixOS - NixOS Wiki

Sorry for that.

So it can access the YubiKey since when I plug it in it shows “success”.
But then it shows me 3 “Authentication failed!” and then “Maximum authentication errors reached”.

My configuration:

boot.initrd = {
    kernelModules = [ "vfat" "nls_cp437" "nls_iso8859-1" "usbhid" ];

    luks = {
      yubikeySupport = true;
      cryptoModules = [ "aes" "xts" "sha512" ];

      devices."nixos-luks" = {
        device = "/dev/nvme0n1p1";

        yubikey = {
          slot = 2;
          twoFactor = false;
          gracePeriod = 30;
          keyLength = 64;
          saltLength = 16;

          storage = {
            device = "/dev/disk/by-label/boot";

The steps to reproduce this:

parted -s /dev/nvme0n1 -- mklabel gpt
parted -s /dev/nvme0n1 -- mkpart primary 512MiB 100%
parted -s /dev/nvme0n1 -- set 1 lvm on
parted -s /dev/nvme0n1 -- mkpart ESP fat32 1MiB 512MiB
parted -s /dev/nvme0n1 -- set 2 boot on

SALT="$(dd if=/dev/random bs=1 count=16 | rbtohex)"
CHALLENGE="$(echo -n $SALT | openssl dgst -binary -sha512 | rbtohex)"        
RESPONSE="$(ykchalresp -2 -x $CHALLENGE)"
KEY="$(echo | pbkdf2-sha512 $((512 / 8)) 1000000 $RESPONSE | rbtohex)"

echo -n "$KEY" | hextorb | cryptsetup luksFormat /dev/nvme0n1p1--cipher="aes-xts-plain64" --key-size="512" --hash="sha512"
echo -n "$KEY" | hextorb | cryptsetup luksOpen /dev/nvme0n1p1 nixos-luks

vcreate /dev/mapper/nixos-luks
vgcreate nixos-lvm /dev/mapper/nixos-luks
lvcreate -L 4G -n swap nixos-lvm
lvcreate -l 100%FREE -n nixos nixos-lvm
mkfs.ext4 -L nixos /dev/nixos-lvm/nixos
mkswap -L swap /dev/nixos-lvm/swap
mkfs.fat -F 32 -n boot /dev/nvme0n1p2
1 Like

Outstanding. Now we’re moving forward.

  1. I assume you ran pvcreate /dev/mapper/nixos-luks instead of vcreate /dev/mapper/nixos/luks.
  2. I see you format the ESP partition. Did you copy the salt and iterations over to /dev/disk/by-label/boot like in Step 5 [1]
  3. To bisect whether the problem is a misconfigured YubiKey/LUKS or a misconfigured boot process, can you manually run through Wiki steps 4-5 of Set up the YubiKey [2] and step 1 of LVM setup [3]. Does that open the LUKS volume, and is it repeatable? If the manual process works multiple times, then I’d suspect the boot sequence isn’t running the same steps you ran manually.

[1]: GitHub - sgillespie/nixos-yubikey-luks: Set up a LUKS-encrypted filesystem for Yubikey in NixOS
[2]: Yubikey based Full Disk Encryption (FDE) on NixOS - NixOS Wiki
[3]: Yubikey based Full Disk Encryption (FDE) on NixOS - NixOS Wiki

For the first point, yes I made a typo.
Then yes I didn’t include that but I’m writing to /boot/crypt-storage and it seems to find it.

I repeated all the steps manually and it stills shows “Authentication failed!”

I repeated all the steps manually and it stills shows “Authentication failed!”

Seems like something may be misconfigured with the YubiKey or LUKS then.

What version of the YubiKey do you have? Looks like there are a variety of them to choose from, and since I’m not familiar with them, perhaps this encrypted disk tutorial is using a feature not available on your model?

From reading about the YubiKey 5, it appears you can create an OTP on slot 2. I wonder if maybe you configured slot 2 with OTP at some point, and it’s currently locked (I don’t know if that’s a thing). The tutorials seem clear, so I’m grasping at straws to figure out why it won’t work for you.

If you run five trials against your YubiKey with a single challenge, for example
ykchalresp -2 -x "hello world" 2>/dev/null, do you get the same response every time, or does it change with each run? If it changes, then maybe that slot thinks it’s supposed to do OTP instead of chal-resp + chal-hmac.

I tested all that and even made sure that the key generated on the new laptop was the same on another one but it is still not working and I have no clue why.