Systemd-boot + root ZFS + native ZFS encryption with keylocation=file://

Help
#1

I have a working system with keylocation=prompt, but prompt has some obvious limitations. For a server, I don’t want to have to type something in or have a keyboard connected (or SSH, but while I had it enabled for systemd-boot, nmap shows no open port 22). The config matches my laptop (prompt) where ZFS does not automount, and uses mount -t zfs -o zfsutil which nixos-install picks up for hardware.

I have read documentation

One thing that particularly caught my eye is an issue reply (Native encryption - request to allow raw keys on devices, (like USB flash) · Issue #6556 · openzfs/zfs · GitHub) to embed the 64 hex bytes in the sector with a newline that gets around a limitation. This worked for loading & unloading keys in the installer, but didn’t for boot without much of an error. I tried then setting up a /00-zfs-keys directory on a new partition to put in an actual file like the Oracle docs noted. This gave me an error saying the device was missing which would clue me into the mounting order (I tried moving it to /boot/00-zfs-keys on the off chance that would work) and I can’t mount it before root because root is ZFS and the current ZFS will be helpful and try to auto open encrypted partitions. However, all of the keylocation=file:// options do not seem to work for me.

Is there a way better way to get a key from a USB? Also, why isn’t the SSH daemon running when this mounts as an alternative for prompt?

NOTE: I did not follow the OpenZFS guide for NixOS on root, because it involed grub2 which I’ve not used in years, an my laptop has been fine with the prompt option + systemd-boot already. I would assume grub2 wouldn’t help me here but maybe? I will be heading to bed after banging me head against this issue all day.

2 Likes
#2

While working to mount the root file system during boot, the system is in what’s called “stage 1” or “initrd”. Userspace software is run in a tmpfs root until it mounts the root fs at a subdirectory, at which point switch_root is used to make it the root mountpoint and the “stage 2” OS is started.

All this is to say that for ZFS’s keylocation=file://... to work, the path you give it has to be there during stage 1, in the stage 1 tmpfs. NixOS does include a convenient way to do this though. The boot.initrd.secrets option tells NixOS that when it’s setting up the boot loader and the menu entries for all the generations, it should also copy some extra files into the initrd archive so they’re in the stage 1 tmpfs.

  boot.inird.secrets."/foobar" = /path/to/foobar;

This says that while setting up the boot loader, the file at /path/to/foobar should be copied into the initrd so that stage 1 has it at /foobar in the tmpfs. Then ZFS will be able to find it for keylocation=file:///foobar.

Do be aware that this of course means that the key file is written unencrypted to your ESP /boot partition. If you want it to get the key file from a removable USB drive, that would involve some custom code to get it to mount the USB drive in stage 1, get the key file out of it, and unmount it.

2 Likes
#3

Good morning. Thanks for the detailed explanation that went beyond my cursory understanding I managed to cobble together trying to understand how the boot system works. While reading it, my spidey-senses went off though about unencrypted keys just living on /boot which is something to avoid. I’m curious how that’s “convenient” as secrets seems like a footgun for security.

If you want it to get the key file from a removable USB drive, that would involve some custom code to get it to mount the USB drive in stage 1, get the key file out of it, and unmount it.

I’ll try to do my due diligence and read about this, but if you get the opportunity, please link to where I could find more information about this.

Hosted by Flying Circus.