I managed to solve the issue. Ended up doing something much simpler.
I just encrypted both drives with the same passphrase, and set boot.initrd.luks.reusePassphrases to true.
I am very curious why your initial solution with crypttab didn’t work. It seems to indicate the disk wasn’t present. Are you sure there wasn’t a typo in the UUID or something? That’s the only thing I can think of that would cause that…
Because it’s waiting for the device to appear. Devices aren’t present immediately while the system boots up. They can take time. So you have to wait for it to show up, and fail if it times out. That’s what timed out waiting for /dev/disk/by-uuid/<the uuid> means. But if you copied and pasted the same UUID for your working solution, obviously that means it wasn’t a typo, and the device wasn’t present for some other reason.
No, the device that timed out was the physical device, not the mapped virtual device. So it was definitely waiting on that physical device to appear for some reason.
I assume the crypttab solution is more elegant. reusePassphrases feels kind of like a hack. I’m also not sure if it takes way longer like that?? It feels like it does.
@ewichuu I’m not even certain that is a hack, I think it may be default behavior for the kernel, I’ve been using the same sort of setup in arch linux for many years. i.e. I have a system with multiple luks encrypted partition and the system automatically reuses the first password on the rest (*if they are the same). However, those were all ext4 and sometimes xfs or jfs.
On NixOS of late I have been trying ZFS. I ended up using a luks encrypted that contains my root on ZFS, and then I also have a luks encrypted swap partition (no ZFS). I attempted to use native ZFS encryption but I had to type the pass twice, I could not get it to reuse when using native ZFS encryption (maybe someone else here got it to work?).
Using luks instead everyting works, I understand there might be a bit more performance that can be unlocked using native ZFS, with the possible expense of reliability when compared to luks.
One question I have to both native ZFS encrypter and those who put ZFS on luks, do you have any special settings for trim? I know turning it on can make the encryption less secure, but on the other hand turning it off can lock off more of the SSD and keep it from doing its wear-leveling properly.
I tend to combat this by having much smaller portions of the disk encrypted, at the moment I am not encrypting /nix as the way I see it, it is all hashed and checked. However, I realize someone with physical access could swap some of those binaries out on me and they might even go unnoticed for a few updates. Any thoughts? Anyone else notice that SSDs die really fast if you fully encrypt them and disable trim?
As for hibernation and ZFS, I do have a separate luks partition with the swap on it (on an optane drive as this laptop happened to come with one). And I have had no issues so far after hibernating, and even enabled zswap. I haven’t even had to force import any datasets yet, but I’ve only had this working for a couple of months now.
Also, I’ll add in that I used the mountpoint=legacy option when mounting before I ran nixos-generate-config, and that all remain in my hardware-configure.nix
The security concerns of enabling trim with LUKS are extremely minimal and only relevant if your attacker is very sophisticated. Just enable trim. Your SSD will live longer and thank you for it.
Password sharing between ZFS native encryption and LUKS is not currently supported.
I’ll say it again and again: Do not use hibernation with ZFS. Yea, you’ll probably be fine… probably… for who knows how long. But the bugs are known, they are unpredictable, and they will kill your pools. Not to mention NixOS hasn’t even merged the fix for the most glaring and fixable of them (somewhat my bad, but it’s also had merge conflicts for months, and it doesn’t truly fix the issue anyway).
@ElvishJerricco I agree absolutely people enable trim, especially if you are doing full disk encryption.
I am still not doing trim on my home partition, and I have it on a completely separate partition not on my main unencrypted zfs pool, I do manually trim other datasets/pools, and have many unencrypted partitions like /nix, and I would like to hear arguments on why that should be encrypted, I know that some things can fall in there unencrypted like /etc/NetworkManager/system-connections right now I’m comfortable with that.
Should I be sticking that in an encrypted partition as well? And is there anyone else who keeps their home/root/var partition or any other partitions on luks and not zfs native encryption? And do you trim them?
Any un-trimmed partition will be bad for the health and performance of the entire SSD, even other partitions.
As for the other questions…
I just encrypt everything. I don’t particularly see a reason not to. The only thing unencrypted on my systems is the /boot partition.
I have ZFS pools using both native encryption and LUKS. I tend to prefer LUKS for a variety of reasons.
There’s my warning from above:
LUKS gives you free systemd handiness. It can do TPM2 or FIDO2 things out of the box.
LUKS can have multiple keyslots, allowing multiple passphrases or security tokens to be enrolled.
But ZFS native encryption is pretty cool. On the systems where I use it, it is really cool to be able to zfs send --raw for encrypted backups. Of course that’s exactly the use case that’s most likely to cause it to bug out…
The performance difference between the two was negligible when I tested it
ya I know, that’s why I limit it to 50GB out of a 1TB drive, which atm just has my /, /var, and /home partitions on it, and /nix, and /opt are on the unencrypted side.
EDIT: on the performance side of things I would have thought that ZFS doing the encryption at the file level might have given it an edge in performance if not in overall encryption security, but from what I’ve read most people have experienced the same as you in that luks seems pretty performant. When in doubt compare cryptsetup benchmark and your own hardware with something like hdparm -Tt /dev/sdx
^does anyone know of such a 'benchmark` to compare zfs to luks?
Why would it be more secure? LUKS just encrypts every single block. It’s not really possible to be more secure than that. In fact ZFS’s internal metadata is unencrypted, meaning the sizes and locations of files is unencrypted, which is technically a security risk.
Just use fio. Run it on unencrypted ZFS, ZFS with native encryption, and ZFS over LUKS. That’s how I tested it.
ya fio would do that nicely, I will give that a shot next time I have a zfs native encrypted fs to test. Can you share your performance data? totally understand if it’s gone.
So far as block encryption versus ZFS encrypting single files encryption being less secure I would have thought that would be included in the trim conversation, which I think you understand so I am confused as to what is not obvious here?
perhaps I worded it wrongly, to be clear I think that the block encryption is superior and that zfs encrypting files ‘leaks’ the same information that trimming might.
LUKS with trim just reveals which blocks might still be in use. ZFS native encryption reveals the entire internal metadata structure, which reveals which blocks are a part of the same files, what dataset they’re in, etc.. ZFS native encryption leaks more information than LUKS with trim.
EDIT: Granted, it’s for good reason. You can scrub and resilver without loading keys with native encryption, for instance.
excellent point, thank you for the clarification. luks is still superior for those concrened with privacy, zfs might be fine for those merely looking to obfuscate data.
I mean to be clear, that extra leaked information is like… not useful. It’s another thing where any attack utilizing that information would be pretty sophisticated. It’s concerning enough that like, a state-level organization might care. But the vast majority of use cases probably don’t.