Hm, the zvol isn’t appearing automatically. I had an issue on one of my systems recently where I had to make it run udevadm trigger /dev/zd* after the pool was imported for systemd to recognize it, but that issue went away on its own so I assumed it wouldn’t be relevant to anyone else. It could be what’s happening to you though. You could try
# important to use a script here for * expansion
systemd.services.<name>.postStop = ''
udevadm trigger /dev/zd*
'';
(Replace <name> with the name of your ZFS import service. I don’t remember if you’re using a custom service for that or just the stock NixOS one; the stock one is named zfs-import-pegasus)
Once that’s fixed, you’ll run into another problem afterward:
This is not going to work; this option does not get translated into shell code. It has to be a path to an actual file, or a path to a socket it can contact to ask for the key.
Okay, implemented your fix. However, the LUKS volume is still asking for user presence, since FIDO2-HMAC requires it, and setting --fido2-with-user-presence=no will not work. I’m trying to find a way around that. Is there any way to perhaps hijack the setup LUKS uses for fido2 devices and decrypt an age secret using the age-plugin-yubikey identities? I could use pkcs11 instead of FIDO2-HMAC, but that would most likely wipe out the age identities I already have on the yubikey, and fido2 seems to be more future-proof.
I mean, don’t try to hijack it. If you don’t want to use the builtin support for FIDO2 as-is, you can just make your own service that produces a key file however you want, order it before systemd-cryptsetup@credstore.service, and set keyFile to the path that your service outputs the key to. Or you can make a socket-activated service and set keyFile to the path of the socket for this protocol.
I didn’t know this, but FWIW this is probably an intentional design choice. There’s not much value in using a Yubikey for unattended decryption; it’s essentially identical to just having a key file on a USB drive (and indeed that’s something that’s pretty easy to set up).
Well, when I say “hijack”, I mean use whatever method LUKS uses to access the Yubikey during stage 1. Ordering my custom service before systemd-cryptsetup@credstore.service wouldn’t let me access the Yubikey, which is what we may have been having trouble with in our prior conversations. I’m wondering if there’s somewhere I can put a script while the LUKS service is running that would allow me to decrypt the age secret using the identities on the Yubikey.
And regarding having the key file on an external USB, wouldn’t the file need to be in plaintext? … Though if it were encrypted, I could theoretically use age-plugin-tpm to decrypt it, since I wouldn’t be able to use age-plugin-yubikey and I don’t want to keep an identity with the secret itself…
why not? What I’m trying to say is that if systemd’s LUKS plugins for Yubikeys don’t do what you want, then you shouldn’t try to “hijack” them since systemd supports letting you just provide keys however you want with your own method. If you want to use the Yubikey in a different way than systemd’s plugins would, then write your own service that does that different thing and either provides the resulting key at a path you set keyFile to or provides it over a socket that you set keyFile to. There’s no reason this service wouldn’t be able to access the Yubikey the same way the systemd LUKS plugins do.
Yes, that’s my point. A Yubikey just auto-decrypting the LUKS header is doing the same thing as a USB drive with the key in plaintext. It makes it so that you need both the USB device and the drive’s ciphertext to get the plaintext of the drive, and it also makes it so that root can do this operation fully automatically. The only threat model I can think of where these things aren’t fully equivalent is an attacker who gains temporary access to the USB device before gaining access to the drive ciphertext; in that case the attacker can copy the key file and hang onto it until they get the drive ciphertext, which they couldn’t do with temporary access to the Yubikey. But that’s just… not a significant threat model. Are you really planning on setting it up so that the drive and the Yubikey aren’t equally accessible, and in that specific way?
Well, it’s more that I have no idea where to begin implementing the Yubikey access, as I don’t know what store paths to add to the United to get the pcscd service to work, or how LUKS implements its own fido2 access. And I’ve been trying so many things and wiping my SSD so many times I’m starting to worry I won’t have too many writes left once I actually finish setting things up, despite using a 1TB TLC, which I should be able to zero out around 600 times or so.
So I’d prefer if there was a way to hook into a preexisting service instead of creating my own, which would most likely have security flaws as well.
And regarding the key file, I was planning on having the Yubikey on my person when not around the machine, and when I unplug it, it would lock automatically. With the threat model you’ve described, however, using a TPM identity to encrypt a secret that stays on the USB drive could also work, since you wouldn’t be able to decrypt it using the identity without the laptop itself. Though I was hoping to use the agenix files that were already in the nix store… Hmm…
Sorry, I’m not very good at this. Still trying to understand cyber security and cryptography in general.
I don’t know what you were expecting when you said this then:
If you were already willing to make age-plugin-yubikey work, then you could just use that in a custom service to produce a key file, no hijacking necessary.
My point is that this is not meaningfully different than just having a plaintext key file on a USB stick. Just like you can take the Yubikey with you, you can also take that USB stick with you. That achieves basically all the same security benefits. An attacker still needs both the USB stick and the laptop to decrypt the laptop’s drive; the only difference is that they only need temporary access to the USB stick to copy the key off if they’ll have access to the laptop later (note that the opposite is true for both the USB stick and Yubikey methods; if they get temporary access to the laptop, they can clone the drive, and then later access to the USB device lets them decrypt it). But that’s just not a realistic threat model; there is no attacker who’s going to take your USB stick, give it back, and then go steal your laptop.
That doesn’t really address that (very small) disadvantage; the TPM is a whole other axis of security.
I don’t really think you need to worry about this. Modern SSDs are rated to last a couple years of 24/7 write loads. You’re not approaching that at all, even if you’re reinstalling NixOS from scratch on every attempt. Also, if I were you I wouldn’t be reinstalling NixOS every time, I’d just be iterating on the same NixOS installation.
I am, most of the time, but the partitioning can get a little complicated at times, and if I change some of the LUKS authentication methods, I have to recreate the volume itself, which sometimes doesn’t happen if I just use the format mode of disko.
There’s nothing in LUKS that requires you to recreate the volume to change. That might be a limitation of disko, and one that you can workaround by just using cryptsetup or systemd-cryptenroll or whatever manually. But LUKS lets you completely change the key slots without wiping it.
Yeah, the attacker only needing the USB for a short time is what I’m afraid of, though how scared I should be, I’m not sure. At least with the Yubikey if I can’t find it, I don’t have to worry about whether or not I have to rotate all my keys; I’d just do it. Little less stressful.
Regarding using age-plugin-yubikey, the issue was setting up the pcscd daemon, which the plugin needs to function. Your advice initially was to copy the preexisting service structure for the daemon to stage 1 and order it before the service that loads the zfs keys; to implement this, do I need to add anything specific to the initrd store paths, such as the the pcsclite package? Additionally, I won’t be able to access /var/lib or the root filesystem in stage 1; could I create a tmpfs for them?
There isn’t much in the way of documentation for a completely user-free yubikey boot, it seems, which is understandable.
Got it. I’ll keep that in mind next time. The problem is that disko doesn’t really allow you to hook into their various mount stages, so I can’t exactly pause after mounting just the zpool with it. I’ll look into what commands they use to mount the pool and copy those in my bootstrapping script.
Wait, this makes even less sense now that I think about it; if the plan is to keep the Yubikey with yourself and not leave it with an unattended machine, what’s the point of disabling user presence and booting unattended? If the Yubikey is always going to be with you, then you’ll be there whenever user presence is needed, and user presence should be no problem.
If you want unattended boot, you probably want measured boot using the TPM2, which lanzaboote and systemd-cryptenroll have very good support for. Part of my point about the USB stick being mostly the same as a Yubikey without user presence for this purpose is that those are both sorta bad options; measured boot is a much better way to achieve that.
I’m not sure what exactly you’ll need; probably just the binaries it refers to in storePaths and the config file you need. Not sure why you think you need /var/lib?
Mostly for convenience reasons; I sometimes stop paying attention for a few minutes and then realize that I needed to press the contact to continue the bootup, which wastes even more time. I thought the Yubikey without user presence would be a decent compromise between always having the key on the machine itself via the TPM, for unattended boot, and using a password, where I always need to be there.
As I understand it, lanzaboot is a secure boot thing, right? For me that would be an extra password or ISO enrollment in order to use a freshly created ISO. I know it seems a bit lazy, but my disability makes reading and understanding a bit difficult and headache-inducing, so the fewer things for me to remember, the better.
Regarding why I need /var/lib, the pcscd daemon looks for its drivers there, if I recall correctly, and there doesn’t seem to be a way to change it…? It’s from this GitHub issue:
Like I had said in the other thread, I’d just try to recreate the normal stage 2 service that nixos normally ships. I would not try to copy from some random script on a github issue. That one is doing several weird things. It’s using runtimeShell for something super weird. It’s checking for another pcscd process to kill because activation scripts get run on every nixos-rebuild switch, so it needs to kill the one running in the regular systemd service or from previous activations (both of which are reasons this stuff just shouldn’t be activation scripts; so it can use the regular service). And it’s using /var/lib for seemingly no good reason; the regular service just sets an environment variable.
The normal service in nixos is super simple. Porting that to stage 1 should be pretty simple and will actually make it work normally. The NixOS options for systemd in stage 1 are intentionally almost identical for systemd in stage 2; the stage 2 module should only need some minor adaptations.
Well you’re always going to need a recovery password no matter what hardware key you use. If you lose the hardware key, you lose the drive, unless you have a recovery key slot. So when you need to boot a NixOS ISO, you can just disable secure boot from the BIOS menu (which, if you did measured boot right, will break the TPM2 unattended unlock), boot the ISO, and use the recovery passphrase. I understand that the recovery passphrase is a burden, but I just don’t think it’s avoidable in any case; you always need to be able to recover the drive if the hardware key is toast. systemd-cryptenroll has an option for generating a super long recovery passphrase, and I recommend using that and storing that passphrase somewhere safe, like a password manager, so you don’t lose it and don’t have to remember it. I would make that recommendation regardless of what hardware key you want to use.
I will caution that doing measured boot right is kinda hard. There’s a bunch of ways like this one to misconfigure it to be easily bypassed. That’s why I don’t often recommend it; you have to know what you’re doing pretty well.
Honestly I can’t really spend more time helping you with this. But my practical recommendations are to keep it simple and do one of:
Use FIDO2 and just let it use user presence; your use case is pretty compatible with this.
Just put a key file on a USB stick. It’s as simple as keyFile = "/key:/dev/disk/by-uuid/the-usb-drive"; (might need to add initrd drivers for USB devices with boot.initrd.availableKernelModules)
Use measured boot. This is definitely the hardest option, but it’s the only good option for unattended boot. Lanzaboote has some features for measured boot, but you do still need to be aware of the article I linked and have a way to prevent that issue. I’m not a fan of the solution described in the article; I think the simplest thing to do is to use crypttabExtraOpts = [ "fixate-volume-key=..." ]; (though determining that value is annoying), as long as you make sure you only refer to the virtual device by /dev/mapper/foo rather than anything like /dev/disk/by-label/foo (disko probably does this the way that won’t work for this). Like I said, this option is hard. But it is the only option for unattended boot that’s actually good.
In any of these options, you’ll want to save a recovery passphrase somewhere safe, generated and enrolled on the disk with systemd-cryptenroll --recovery-key
Thanks again for all the help. Got everything working. Now just need to trim the fat. Unfortunately, also had to use the method linked in the GitHub issue instead of the pcscd service since I was running into issues with the pcscd user and group not existing during stage 1.
I think I mentioned in one of these threads that you would need to either set the serviceConfig.User / serviceConfig.Group to root (adding users in stage 1 is a bad idea) or set serviceConfig.DynamicUser = true;. I’m telling you, what that script from the github thing is doing is just the same thing as the upstream service except needless hacked together and worse.
