Bumping to 26.05, `system-boot` fails to boot my system

Help
1

The system I have used for years, running on stable, looked like the following:

{
  boot.loader = {
    efi.canTouchEfiVariables = true;
    systemd-boot.enable = true;
  };

  boot.initrd.luks.devices = {
    cryptkey.device = "/dev/nvme0n1p3";

    cryptroot = {
      device = "/dev/nvme0n1p5";
      keyFile = "/dev/mapper/cryptkey";
    };

    cryptswap = {
      device = "/dev/nvme0n1p4";
      keyFile = "/dev/mapper/cryptkey";
    };
  };

  fileSystems."/boot" = {
    device = "/dev/disk/by-uuid/<UUID>";
    fsType = "vfat";
  };

  fileSystems."/" = {
    device = "/dev/disk/by-uuid/<UUID>";
    fsType = "ext4";
  };

  swapDevices = [
    { device = "/dev/disk/by-uuid/<UUID>"; }
  ];
}

Wanting to update, I read the incompatibilities notes in the documentation: Appendix B. Release Notes , and therefore updated to the following…

fileSystems."/" = {
  device = "/dev/mapper/cryptroot";
  fsType = "ext4";
};

However, the resulting system fails to boot; I observe a hang up while these messages (alternating - order might not be right) are displayed:

(1 of 2) A start job is running for /dev/mapper/cryptkey
(2 of 2) A start job is running for /dev/mapper/cryptroot

I tried to also update:

swapDevices = [
  { device = "/dev/mapper/cryptswap"; }
];

to no avail.

Reading Full disk encryption with ordering - #2 by ElvishJerricco , it seemed to me that this use case should be working out of the box…
What am I doing wrong?

2

Please see nixos/boot: system no longer boots with systemd stage 1 · Issue #527478 · NixOS/nixpkgs · GitHub and Correcting my NixOS LUKS mistakes | Ivan Petkov

The TL;DR is to use a file in a filesystem on /dev/mapper/cryptkey, not /dev/mapper/cryptkey itself.

Systemd upstream explicitly does not want to be able to handle the case where a raw block device (here /dev/mapper/cryptkey) from LUKS is used as a keyFile or any kind of dependency.

In the past, this was patched into NixOS (to remove the udev rule that excludes this). However, Nixpkgs is generally against these kind of “add feature” patches so that we can follow upstream faster, to get various fixes faster. So this was removed in 26.05.

This item in the 26.05 release notes applies to all cases where such /dev/mapper/* devices are used as a dependency. I will admit that this is a rather obscure interaction.

  • Removed the ability from systemd to depend on open-yet-unformatted dm-crypt
    devices as systemd device units. For example you cannot call systemd-makefs
    on such devices. This was never implemented upstream but patched into
    Nixpkgs’s systemd. Removing this allows us to drop a custom systemd patch
    improving systemd maintainability and update speed.

An interesting thing that this seems to be an extremely specific setup: Every single user I’ve seen that has run into this has exactly the devices /dev/mapper/cryptroot and key /dev/mapper/cryptkey.

2 Likes
3

This piqued my interest - it suggests to me that all these users may have followed a commonly-used setup guide, so we could suggest a 26.05-supported revision to the author to help out those who originally followed it as well as future users who may try it out.

I found Installing NixOS and ZFS on my Desktop | Ivan Petkov , and it looks like the author has done exactly that already:

Note: the original version of this write-up […] will fail to boot starting with NixOS 26.05. This has since been corrected […], but for anyone who may have followed the earlier instructions, see here on how to correct the issue

But perhaps there are more similar guides or “forks” of this one which we can also provide suggestions to.

4

Almost 10 years ago, I took inspiration on the setup with LUKS from @grahamc (c.f. NixOS on a Dell 9560 - Graham Christensen), and I documented what I did (mostly for myself) on NixOS on a Dell XPS15 9560 | pamplemousse’s blog .

1 Like