Unlocking multiple luks devices with same passphrase

Help
1

I want to unlock all my luks devices/partitions by typing the passphrase only once during boot.

When I installed NixOS I used the built-in partition manager to set up the partitions as btrfs encrypted with luks (except /boot). Afterwards I manually set up a data disk as a luks device with a btrfs filesystem.
I’ve adjusted my configuration.nix, but I’m being asked twice for the passphrase during boot, even though I explicitly set boot.initrd.luks.reusePassphrases to true.

I’ve tried to set preLVM = true; but that didn’t change anything.
The passphrases are the same for all the encrypted luks devices.

How to make NixOS reuse the typed passphrase during boot?

lsblk (irrelevant devices removed)

NAME                                          UUID                                 MOUNTPOINT
sda                                           e94541c7-7111-46c8-a1f6-3347c6761191 
└─data                                        ed7fc4d7-944a-48e5-854a-2eb9d0c54ee0 /mnt/data                                                                       
nvme1n1                                                                            
├─nvme1n1p1                                   1546-8FD5                            /boot
├─nvme1n1p2                                   8be8b25f-42a1-41c4-ab25-d0c40b958efd 
│ └─luks-8be8b25f-42a1-41c4-ab25-d0c40b958efd 7bf3cc97-e149-4be7-a161-c623780fec07 /nix/store
└─nvme1n1p3                                   bfdf62d2-38eb-4e79-bddf-3b0fcbfa2063 
  └─luks-bfdf62d2-38eb-4e79-bddf-3b0fcbfa2063 24e15ca0-e03f-4c9b-8c38-99a916264600 [SWAP]

journalctl -b 0

stage-1-init: Passphrase for /dev/disk/by-uuid/8be8b25f-42a1-41c4-ab25-d0c40b958efd:
kernel: Key type encrypted registered
stage-1-init: Verifying passphrase for /dev/disk/by-uuid/8be8b25f-42a1-41c4-ab25-d0c40b958efd... - success
stage-1-init: Passphrase for /dev/disk/by-uuid/bfdf62d2-38eb-4e79-bddf-3b0fcbfa2063: reused
stage-1-init: Verifying passphrase for /dev/disk/by-uuid/bfdf62d2-38eb-4e79-bddf-3b0fcbfa2063... - success
stage-1-init: Starting device mapper and LVM...
kernel: BTRFS: device label nixroot devid 1 transid 921 /dev/mapper/luks-8be8b25f-42a1-41c4-ab25-d0c40b958efd scanned by btrfs (700)
stage-1-init: Scanning for Btrfs filesystems
stage-1-init: Registered: /dev/mapper/luks-8be8b25f-42a1-41c4-ab25-d0c40b958efd
stage-1-init: Passphrase for /dev/disk/by-uuid/e94541c7-7111-46c8-a1f6-3347c6761191:
stage-1-init: Verifying passphrase for /dev/disk/by-uuid/e94541c7-7111-46c8-a1f6-3347c6761191... - success
stage-1-init: Mounting /dev/disk/by-uuid/7bf3cc97-e149-4be7-a161-c623780fec07 on /...
kernel: BTRFS info (device dm-0): First mount of filesystem 7bf3cc97-e149-4be7-a161-c623780fec07
kernel: BTRFS info (device dm-0): Using crc32c (crc32c-intel) checksum algorithm
kernel: BTRFS info (device dm-0): Using free space tree
kernel: BTRFS info (device dm-0): Enabling ssd optimizations
unknown: Booting system configuration /nix/store/7yig8lxp471pzvrf5p3g19qxsxj4153p-nixos-system-taurus-23.11.6981.27c13997bf45

/etc/nixos/configuration.nix

{ config, pkgs, ... }:

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
	<home-manager/nixos>
    ];

  # Bootloader
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  # LUKS

  ## Swap
  boot.initrd.luks.devices."luks-bfdf62d2-38eb-4e79-bddf-3b0fcbfa2063".device = "/dev/disk/by-uuid/bfdf62d2-38eb-4e79-bddf-3b0fcbfa2063";
  
  ## Data drive
  boot.initrd.luks.reusePassphrases = true;

  boot.initrd.luks.devices = {
    data = {
      device = "/dev/disk/by-uuid/e94541c7-7111-46c8-a1f6-3347c6761191";
      preLVM = false;
    };
  };

  # File systems
  fileSystems."/mnt/data" = {
    device = "/dev/disk/by-uuid/ed7fc4d7-944a-48e5-854a-2eb9d0c54ee0"; 
    fsType = "btrfs"; 
  };

/etc/nixos/hardware-configuration.nix

# Do not modify this file!  It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations.  Please make changes
# to /etc/nixos/configuration.nix instead.
{ config, lib, pkgs, modulesPath, ... }:

{
  imports =
    [ (modulesPath + "/installer/scan/not-detected.nix")
    ];

  boot.initrd.availableKernelModules = [ "nvme" "xhci_pci" "ahci" "usb_storage" "usbhid" "sd_mod" ];
  boot.initrd.kernelModules = [ ];
  boot.kernelModules = [ "kvm-amd" ];
  boot.extraModulePackages = [ ];

  fileSystems."/" =
    { device = "/dev/disk/by-uuid/7bf3cc97-e149-4be7-a161-c623780fec07";
      fsType = "btrfs";
      options = [ "subvol=@" ];
    };

  boot.initrd.luks.devices."luks-8be8b25f-42a1-41c4-ab25-d0c40b958efd".device = "/dev/disk/by-uuid/8be8b25f-42a1-41c4-ab25-d0c40b958efd";

  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/1546-8FD5";
      fsType = "vfat";
      options = [ "fmask=0022" "dmask=0022" ];
    };

  swapDevices =
    [ { device = "/dev/disk/by-uuid/24e15ca0-e03f-4c9b-8c38-99a916264600"; }
    ];

  # Enables DHCP on each ethernet and wireless interface. In case of scripted networking
  # (the default) this is the recommended approach. When using systemd-networkd it's
  # still possible to use this option, but it's recommended to use it in conjunction
  # with explicit per-interface declarations with `networking.interfaces.<interface>.useDHCP`.
  networking.useDHCP = lib.mkDefault true;
  # networking.interfaces.enp6s0.useDHCP = lib.mkDefault true;
  # networking.interfaces.wlp5s0.useDHCP = lib.mkDefault true;

  nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
  hardware.cpu.amd.updateMicrocode = lib.mkDefault config.hardware.enableRedistributableFirmware;
}
2

Usually you’d put the key for the subsequent disks into the first one that gets unlocked. It feels like that’s a safe way to store the second key, given that everything is encrypted when locked, but it does mean having a plaintext file hanging around

3

you need both luks devices in a section like this:

  boot.initrd.luks.devices = {
    "cryptswap".device = "/dev/disk/by-uuid/deadbeef1";
    "cryptroot".device = "/dev/disk/by-uuid/deadbeef2";
  };

or defined separately:

boot.initrd.luks.devices.“cryptswap”.device = “/dev/disk/by-uuid/deadbeef1”;
boot.initrd.luks.devices.“cryptroot”.device = “/dev/disk/by-uuid/deadbeef2”;

4

@thoth they have that already.

@opteron What you have looks fine to me. I would have expected that to work. Maybe try systemd initrd? boot.initrd.systemd.enable = true;, although you’ll have to change the fileSystems.<mountpoint>.device definitions to use /dev/mapper/ paths instead of /dev/disk/by-uuid/ paths because… reasons.

5

Not exactly, he has the second device in a data block, unlike the first:

“Not Found” →
https://search.nixos.org/options?channel=23.11&from=0&size=50&sort=relevance&type=packages&query=boot.initrd.luks.devices.data

the proper way to do this preLVM thing would be

boot.initrd.luks.devices.“cryptroot”.preLVM = “/dev/disk/by-uuid/deadbeef2”;

https://search.nixos.org/options?channel=23.11&show=boot.initrd.luks.devices.<name>.preLVM&from=0&size=50&sort=relevance&type=packages&query=preLVM

6

data is the name of the luks volume. What they have is equivalent to:

boot.initrd.luks.devices = {
  "luks-8be8b25f-42a1-41c4-ab25-d0c40b958efd".device = "/dev/disk/by-uuid/8be8b25f-42a1-41c4-ab25-d0c40b958efd";
  "luks-bfdf62d2-38eb-4e79-bddf-3b0fcbfa2063".device = "/dev/disk/by-uuid/bfdf62d2-38eb-4e79-bddf-3b0fcbfa2063";
  "data".device = "/dev/disk/by-uuid/e94541c7-7111-46c8-a1f6-3347c6761191";
  "data".preLVM = false;
}

So it’s just that the third one has a name that isn’t the same style as the first two. (Also note one of those is in hardware-configuration.nix and not configuration.nix)

7

ok I see it correctly now, but this is currently working for me without issues.

@opteron if it’s any help here is what my boot section looks like:

  boot = {
    # Enable binfmt emulation of aarch64-linux.
    binfmt.emulatedSystems = [ "aarch64-linux" ];
    tmp = {
      useTmpfs = true;
    };
    zfs = {
      forceImportRoot = false;
      allowHibernation = true;
    };
    plymouth = {
      enable = true;
      theme = "bgrt";
      font = "${pkgs.dejavu_fonts.minimal}/share/fonts/truetype/DejaVuSans.ttf";
    };
    resumeDevice =  "/dev/disk/by-uuid/deadbeef3";
    #kernel.sysctl.vm.swappiness = 33; 
    kernelParams = [
      "zswap.enabled=1"
      "zswap.compressor=zstd"
      "zswap.max_pool_percent=8"
      "zswap.zpool=z3fold"
      "zswap.accept_threshold_percent=88"
      "i915.force_probe=9a49"
      "resume=UUID=deadbeef3"
    ];
    kernelModules = [
      "lz4"
      "zstd"
      "nfs"
      "z3fold"
      "kvm-intel"
    ];
    loader = {
      timeout = 1;
      efi.canTouchEfiVariables = true;
      grub = {
        enable = true;
        configurationName = "NixOS";
        configurationLimit = 15;
        device = "nodev";
        useOSProber = true;
        zfsSupport = true;
        efiSupport = true;
        fontSize = 36;
        mirroredBoots = [
          { devices = [ "nodev"]; path = "/boot"; }
        ];
      };
    };
    initrd = {
      supportedFilesystems = [
        "nfs"
        "zfs"
      ];
      luks.devices = {
        "cryptstorage".device = "/dev/disk/by-uuid/deadbeef1";
        "cryptroot".device = "/dev/disk/by-uuid/deadbeef2";
      };
      availableKernelModules = [
        "lz4"
        "nfs"
        "rtsx_pci_sdmmc"
        "sd_mod"
        "usb_storage"
        "thunderbolt"
        "vmd"
        "xhci_pci"
        "zstd"
        "z3fold"
      ];
      kernelModules = [
        "aesni_intel"
        "cryptd"
        "dm-snapshot"
        "lz4"
        "nfs"
        "nvme"
        "z3fold"
        "zstd"
      ];
    };
  };
8

@thoth You were right. I changed my hardware-configuration.nix to this:

  boot.initrd.luks.devices = {
    "cryptroot".device = "/dev/disk/by-uuid/8be8b25f-42a1-41c4-ab25-d0c40b958efd";
    "cryptswap".device = "/dev/disk/by-uuid/bfdf62d2-38eb-4e79-bddf-3b0fcbfa2063";
    "cryptdata".device = "/dev/disk/by-uuid/e94541c7-7111-46c8-a1f6-3347c6761191";
  };

and it works as expected!

# Do not modify this file!  It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations.  Please make changes
# to /etc/nixos/configuration.nix instead.

I’m unsure if it’s okay to modify hardware-configuration.nix manually, but I don’t know how to get all luks devices in one section otherwise :thinking:

9

I’m certain that @ElvishJerricco was, indeed, correct at the highest level, given his seniority!

However, I’m also curious as to what really was the root of the issue here.

I’m pretty new to NixOS myself, and you can see that given the corrections @ElvishJerricco gave. He is certainly correct in how the ‘derivation’ should go.

I think in this instance (and those similar to my own) I need to learn the commands to dump out the ‘derivation’ to examine the differences.

Anyhow, I’m very glad you got it working and please report back if you figure out “what” exactly went wrong on your first attempt. But don’t spend too much time on that, just go forward and enjoy Nix. (and commit everything to some sort of SCM, e.g. git)

10

@thoth I think the problem was that the luks unlocking “happened” in different sections or in “different files” (configuration.nix and hardware-configuration.nix).
I assumed NixOS just collected all the expressions from configuration.nix and imports and then figured it all out, but apparently boot.initrd.luks.devices.<name>.device needs to be in the same section (and in the same file) to achieve boot.initrd.luks.reusePassphrases.

I’m working on that. Learned the basics of Git, but the permissions of /etc/nixos/configuration.nix makes it not straightforward unless I want to run git as sudo. I’ll have to look into that. I do really enjoy Nix though!

11

As I understand it, and as @ElvishJerricco pointed out nix should have merged that data correctly. Nix compiles these files into some ‘derivation’ that is the sum total of all those files and individual data structures into a single data structure, that I would like to ‘prettyprint’. There is some difference in that derivation in how you had it before that would identify the issue. Again, though, just go forward and commit often!

12

I learned alot from Will Taylor’s youtube series on Nix, though it is a bit dated at this point.

I’m a N00B as well to Nix so please take my advice with a grain of salt.

But I have a repo here that represents where I went from Will’s setup.

In this fashion all your nix files live in your home directory once you get a machine ‘bootstrapped’, and you apply your current setup with something like “sudo nixos-rebuild switch --flake .#”

or with my script nx as

I chose to use ~/.nx, of which I have an example here.

Again, I’m a n00b at NixOS, but not at git! I welcome PRs :wink:

13

This is really not correct. First of all, in one file, multiple “sections” is just syntactic sugar for literally (I mean, actually, literally) the exact same thing as one “section”. It’s just attrset syntax. Across files, you’re relying on the NixOS module system to merge the definitions, which it very much does do. So what you had really should have worked. Based on the log you posted originally, it looks like it did reuse the passphrase for one disk, and then it didn’t for the third.

I think trying to manually set preLVM is actually what went wrong. I just looked at luksroot.nix, and it looks like the postCommands code unmounts the ramfs that the password to reuse gets saved in, and that gets run on both the preLVM devices and the regular devices, meaning passphrases are lost across the preLVM boundary. That should probably be considered a NixOS bug. We actually have an issue open about using the kernel keyring instead of a ramfs, which, yea, would have avoided this.

@thoth This actually isn’t really a matter of derivations. Really, it comes down to the NixOS module system. This is just an eval-time thing, and doesn’t inherently involve derivations. Basically, the module system looks at all your modules (i.e. imports = [ /* ... */ ];) and merges the definitions in them into a final config value. This is the single source of truth of your configuration. So all your boot.initrd.luks.devices definitions get merged together into a single config.boot.initrd.luks.devices attribute set. Then the luksroot.nix module sets some different definitions based on that single source of truth.

14

@ElvishJerricco Thanks for the explanation!

My vocabulary is probably wrong when I said ‘derivation’, but this single source of truth of which you speak is what I’m getting at.

Is there a way to look at this final merged config ‘value’?