tl;dr: suspend-to-disk works but bringing system back up fails, seemingly due to trying to use swap before it has been decrypted.
Hi, I’ll do my best to explain my environment and goals so bear with me.
Environment:
- systemd initrd
- encrypted root and swap partitions sharing a key which exists as an ordinary file on a vfat flashdrive (travel machine, so this way I don’t have to manually punch in a long password but can instead just keep the key on my keychain)
- at present no LVM layer
Here’s the relevant config:
{ config, lib, pkgs, modulesPath, ... }:
{
imports = [
(modulesPath + "/installer/scan/not-detected.nix")
];
boot = {
loader = {
systemd-boot.enable = true;
efi.canTouchEfiVariables = true;
timeout = 0; # Note: to pull up the boot menu, hold space during startup
};
initrd = {
availableKernelModules = [
"xhci_pci"
"thunderbolt"
"vmd"
"ahci"
"nvme"
"usb_storage"
"sd_mod"
"rtsx_usb_sdmmc"
];
kernelModules = [
"uas"
"usbcore"
"usb_storage"
"vfat"
"nls_cp437"
"nls_iso8859_1"
];
systemd.enable = true;
systemd.tpm2.enable = false;
systemd.mounts =
let
USBLABEL = "EMTEC\\x20D250"; # note: the label is actually "EMTEC D250"
in
[
{
what = "/dev/disk/by-label/${USBLABEL}";
where = "/key";
options = "ro";
type = "vfat";
}
];
luks.devices."cryptroot" = {
device = "/dev/disk/by-uuid/e942614b-ab3e-4891-b7a6-47e4df7a1593";
keyFile = "/key/path/to/keyfile";
};
luks.devices."cryptswap" = {
device = "/dev/disk/by-uuid/1852870d-2680-4355-9722-ba4d04305b99";
keyFile = "/key/path/to/keyfile"; # same exact path as for cryptroot
};
};
kernelModules = [ "kvm-intel" ];
extraModulePackages = [ ];
};
fileSystems."/" = {
device = "/dev/mapper/cryptroot";
fsType = "ext4";
};
fileSystems."/boot" = {
device = "/dev/disk/by-uuid/62A9-8259";
fsType = "vfat";
options = [ "umask=0077" ];
};
swapDevices = [
{
device = "/dev/mapper/cryptswap";
}
];
nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
hardware.cpu.intel.updateMicrocode = lib.mkDefault config.hardware.enableRedistributableFirmware;
}
Goals:
I’d like to continue to use systemd initrd for a number of reasons, but I’m trying to get hibernation to work such that all I have to do after the suspend-to-disk is plug in my USB drive and hit the on button. I don’t particularly care whether the solution involves a swap partition, swap file, or whatever else though less work would be better.
Current behavior:
Suspending to disk works, and on resume it correctly reads from efi variables the path of the swap file. However it then freezes with a timeout of 2 minutes, apparently waiting for the swapfile to become available as it is encrypted. I’m unsure of whether the usb drive gets mounted at this time or not. When the two minutes are over, it gives up and proceeds with a normal boot which goes to plan (the decryption key is correctly read from the usb drive in order to access cryptroot). After boot, swapon --show outputs
NAME TYPE SIZE USED PRIO
/dev/dm-1 partition 14,9G 0B -2
Other things I’ve done:
Reading the man pages has so far not been as helpful as I might like. crypttab(5) mentions an option x-initrd.attach but the wording of the man page in addition to errors from nix make it unclear if it applies to swap. bootup(7) does have a nice full explanation of the bootup process, but x-initrd.attach is not mentioned. x-initrd.mount (in crypttab(5) the attach option is described as similar to the mount one) is, however it appears to be used in the initrd after initrd-root-device.target. I can only assume this occurs after the decision is made about whether or not to resume from swap, as the man page states that:
...Before any file systems
are mounted, the manager will determine whether the system shall resume from hibernation or proceed with normal boot.
This is accomplished by systemd-hibernate-resume.service which must be finished before local-fs-pre.target, so no
filesystems can be mounted before the check is complete. When the root device becomes available,
initrd-root-device.target is reached. If the root device can be mounted at /sysroot, the sysroot.mount unit becomes
active and initrd-root-fs.target is reached
systemd-hibernate-resume.service isn’t on the handy-dandy diagram, but if I’m reading this right it must be executed somewhere between basic.target and initrd-root-fs.target, i.e. before x-initrd.mount and (I guess??) x-initrd.attach have any relevance. I haven’t found any man page that explains the bootup process in the case of hibernation restoration.
The other confusing thing is based on the output/logs it appears that disk decryption occurs after the decision regarding hibernation, i.e. in the initrd, however cryptsetup.target is all the way at the beginning of the bootup, and appears to be completed before sysinit.target even. It’s also unclear how that interacts with swap.target when you have swap encryption, not to mention swapfiles.
Speaking of which, my first attempt at this config involved a swapfile in /var/lib/swapfile but was ultimately unsuccessful. In desperation I changed to a separate swap partition on the theory that that might be a more “normal” setup, and/or it might need to be able to read swap before cryptroot is available however that didn’t seem to change anything.
Other notable things I tried:
- setting up swap encryption via swapdevices.*.encrypted (I think it’s just an alias, but nixos has some funny quirks)
- setting neededForBoot to true on the keyfile mount (didn’t change anything, presumably because that just makes it available by the initrd.target or something)
- getting to a shell during boot at the critical point to have a poke around: didn’t work, the shell was only available after some point later (maybe initrd.target, not sure).
- I noticed the resume target used the by-uuid path, so I tried forcing the /dev/mapper path via kernel parameter in the hopes that that would cause the initrd to realise it needed to unlock that luks device but that also didn’t work.
I’d be grateful for any advice that came my way. Thanks in advance.