Trying to create a config for LUKS+BTRFS+TMP2

Help
1

Hi

I want to install Nixos on my work notebook, but for that I need to encrypt the disk. I am testing it on a Poxmox VM, I do the formatting, I do the Nixos flake installation, and lastly I enroll the passphrase, and everything seems fine. But no matter what I do, during the boot it always asks me for the passphrase. I never used LUKS before and am kinda lost here. Not even sure if this stuff properly works on Proxmox. I would be super grateful for help.

My hardware configuration:

# Do not modify this file!  It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations.  Please make changes
# to /etc/nixos/configuration.nix instead.
{ config, lib, pkgs, modulesPath, ... }:

{
  imports =
    [ (modulesPath + "/profiles/qemu-guest.nix")
    ];

  boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd" "virtio_pci" "virtio_scsi" "sd_mod" "sr_mod" ];
  boot.initrd.kernelModules = [ "cryptd" ];
  boot.kernelModules = [ "kvm-intel" ];
  boot.extraModulePackages = [ ];

  boot.initrd.luks.devices."luksfs".device = "/dev/disk/by-partlabel/luks";
  boot.initrd.systemd.tpm2.enable = true;
  security.tpm2.enable = true;

  fileSystems."/" =
    { device = "/dev/disk/by-label/NIXOS";
      fsType = "btrfs";
      options = [ "noatime" "compress=zstd" "ssd" "discard=async" "subvol=@" ];
    };

  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/EFI";
      fsType = "vfat";
      options = [ "fmask=0022" "dmask=0022" ];
    };

  fileSystems."/home" =
    { device = "/dev/disk/by-label/NIXOS";
      fsType = "btrfs";
      options = [ "noatime" "compress=zstd" "ssd" "discard=async" "subvol=@home" ];
    };

  fileSystems."/var/log" =
    { device = "/dev/disk/by-label/NIXOS";
      fsType = "btrfs";
      options = [ "noatime" "compress=zstd" "ssd" "discard=async" "subvol=@log" ];
    };

  fileSystems."/nix" =
    { device = "/dev/disk/by-label/NIXOS";
      fsType = "btrfs";
      options = [ "noatime" "compress=zstd" "ssd" "discard=async" "subvol=@nix" ];
    };

  # fileSystems."/var/lib/swap" =
  #   { device = "/dev/disk/by-label/NIXOS";
  #     fsType = "btrfs";
  #     options = [ "nodatacow" "subvol=@swap" ];
  #   };
  #
  # swapDevices = [{
  #   device = "/var/lib/swap/swapfile";
  #   size = 32*1024; # 32 GB
  # }];

  # Enables DHCP on each ethernet and wireless interface. In case of scripted networking
  # (the default) this is the recommended approach. When using systemd-networkd it's
  # still possible to use this option, but it's recommended to use it in conjunction
  # with explicit per-interface declarations with `networking.interfaces.<interface>.useDHCP`.
  networking.useDHCP = lib.mkDefault true;
  # networking.interfaces.ens18.useDHCP = lib.mkDefault true;

  nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
}

And my formating script:

#!/usr/bin/env bash

if [ "$EUID" -ne 0 ]; then
  echo "Error: This script must be run as root."
  exit 1
fi

if mountpoint -q /mnt; then
    echo "Error: Something is already mounted on /mnt."
    exit 1
fi

if [ -n "$(ls -A /mnt)" ]; then
    echo "Error: The /mnt directory is not empty."
    exit 1
fi

if [ "$#" -ne 1 ]; then
    echo "Usage: $0 <disk_path>"
    echo "Example: $0 /dev/nvme0n1"
    exit 1
fi

DISK="$1"

if [ ! -b "$DISK" ]; then
    echo "Error: '$DISK' does not exist or is not a block device."
    exit 1
fi

DEVICE_TYPE=$(lsblk -dno TYPE "$DISK")

if [[ ! "$DEVICE_TYPE" == "disk" ]]; then
  echo "'$DISK' is not a disk!"
  exit 1
fi

wipefs -af $DISK
sgdisk -n 1:2MiB:+1024MiB -t 1:ef00 -c 1:EFI $DISK
sgdisk -n 2:0:0 -t 2:8309 -c 2:"luks" $DISK

cryptsetup luksFormat -q /dev/sda2 --key-file ./password.txt
cryptsetup open /dev/sda2 luksfs --key-file ./password.txt

mkfs.vfat -F32 -n EFI /dev/disk/by-partlabel/EFI
mkfs.btrfs -fL NIXOS /dev/mapper/luksfs

until [ -b /dev/disk/by-label/NIXOS ]
do
     sleep 1
done

mount -o noatime,discard,ssd,nodev /dev/disk/by-label/NIXOS /mnt
btrfs subvolume create /mnt/@
btrfs subvolume create /mnt/@home
btrfs subvolume create /mnt/@log
btrfs subvolume create /mnt/@nix
btrfs subvolume create /mnt/@swap
umount /mnt

mount -o subvol=@,noatime,compress=zstd,ssd,discard=async,space_cache=v2 /dev/disk/by-label/NIXOS /mnt
mkdir -p /mnt/{boot,home,nix,var/{log,/lib/swap}}
mount -o fmask=0022,dmask=0022 /dev/disk/by-label/EFI /mnt/boot
mount -o subvol=@home,noatime,compress=zstd,ssd,discard=async /dev/disk/by-label/NIXOS /mnt/home
mount -o subvol=@log,noatime,compress=zstd,ssd,discard=async /dev/disk/by-label/NIXOS /mnt/var/log
mount -o subvol=@nix,noatime,compress=zstd,ssd,discard=async /dev/disk/by-label/NIXOS /mnt/nix
mount -o subvol=@swap,nodatacow /dev/disk/by-label/NIXOS /mnt/var/lib/swap
nixos-generate-config --root /mnt

sed -i '/swap/ b; s/"subvol/"noatime" "compress=zstd" "ssd" "discard=async" "subvol/g' /mnt/etc/nixos/hardware-configuration.nix
sed -i '/swap/ s/"subvol/"nodatacow" "subvol/g' /mnt/etc/nixos/hardware-configuration.nix

The cmd I use to enroll the passphrase (still on the live system, secure boot in the vm is turned off):

sudo systemd-cryptenroll --wipe-slot=tpm2 --tpm2-device=auto --tpm2-pcrs=0+7 /dev/disk/by-partlabel/luks

(Edit: Boot problem solved, a stupid mistake from my side. The passphrase/TPM problem remains)

2

I never worked with a TPM on Linux but since you are running this inside of a VM, does your VM have access to the TPM?

Does unlocking the disk in the live system via the TPM work?
I assume that is something one can do.

3

The TPM2 stuff only works if you enable systemd initrd with boot.initrd.systemd.enable = true;. We should probably make it an error to enable boot.initrd.systemd.tpm2.enable without it. (Also security.tpm2.enable = true; is not relevant to this feature)

But, please understand this is much more difficult than it seems. You can’t just systemd-cryptenroll the TPM2 and expect this to be secure. You have to take an awful lot of precautions, or else it’s trivial for the disk to be decrypted by an attacker. For instance, the way you’ve done it, an attacker can simply boot another operating system on the machine and decrypt the disk freely. You can partially prevent this by enabling secure boot with something like Lanzaboote for NixOS, but this category of attacks will still make it trivial to decrypt the disk. You have to use some measured boot mechanism to ensure that only a trusted initrd can decrypt the disk, and then make sure that the initrd never hands control to a stage 2 that isn’t trustworthy without first invalidating the measured boot TPM2 state.

This is difficult to do correctly. I strongly recommend just accepting that you have to enter a passphrase during boot, because the risk is that your disk might as well not be encrypted.

5 Likes
4

Hi, yes Proxmox allows the addition of an TPM to a VM.

5

Thanks for your very enlightening comment! I didn’t know about anything of that. In the article you linked there is also another article for Nix laying out how to use Secure Boot together with LUKS and TPM in a reasonably safe way. But this seems to be way too much experimentation for me now, so I guess I just go with the password solution for now.

1 Like