Unlocking Luks Devices at boot using clevis+tang

Help
1

Hi,

I’m currently building a new system, thats booting from zfs on top of two luks encrypted drives.
I got to the point, that scripted stage 1 and systemd stage 1 work by asking for the encryption keys at boot and the system boots up fine.
But I want to use clevis to unlock the drives using secrets from tang. I got this working some time ago using debian, but can’t figure it out on nixos.

I tried unlocking the discs with postDevice commands in scripted stage 1 or custom systemd units in systemd stage 1, but thats fails with

device-mapper: table: 254:0 crypt: unknown target type
device-mapper: ioctl: error adding target to table

So is there a propper way to achieve this?

Regards and thanks

2

There’s a section in the manual about this: NixOS Manual

3

thanks for the hint. This uses a special jwe file. I want to use the secret stored in the devices luks header via

clevis luks bind

is there any option to get this working?

4

I don’t think that’s an option with the current module, but that certainly sounds like a welcome improvement. PRs welcome!

5

Got some kind of progress on this…
Using scripted stage 1 and:

boot.initrd.luks.devices = {
    nvme0crypt = {
      device = "/dev/nvme0n1p2";
      preOpenCommands = ''
      export PATH=$PATH:${pkgs.curl}/bin/:${pkgs.gnused}/bin/
      ${pkgs.clevis}/bin/clevis luks unlock -d /dev/nvme0n1p2 -n nvme0crypt
      '';
   };
};

I get clevis to unlock the drive, but scripted stage 1 does ask for the password and fails because device is already in use.
Any suggestions on this, or should i try to implement/port clevis-systemd as used by debian?

6

What is “clevis-systemd”? And btw the scripted stage 1 is going to eventually be phased out, so it’s preferable to prioritize systemd stage 1.

7

Finally got it to work. Bit hacky but works:

boot.initrd.luks.devices = {
  dummy = {device = "/dev/nonExistent"; crypttabExtraOpts = ["nofail"];};
};
boot.initrd.systemd.storePaths = [ "${pkgs.clevis}" "${pkgs.curl}" "${pkgs.cryptsetup}" "${pkgs.luksmeta}" "${pkgs.gnused}" "${pkgs.tpm2-tools}" "${pkgs.jose}" "${pkgs.libpwquality}" "${pkgs.coreutils}" "${pkgs.gnugrep}" "${pkgs.bash}"];
boot.initrd.systemd.services.unlocknvme0 = {
    path = [ "${pkgs.clevis}" "${pkgs.curl}" "${pkgs.cryptsetup}" "${pkgs.luksmeta}" "${pkgs.gnused}" "${pkgs.tpm2-tools}" "${pkgs.jose}" "${pkgs.libpwquality}" "${pkgs.coreutils}" "${pkgs.gnugrep}" "${pkgs.bash}"];
    enable = true;
    unitConfig = {Description = "Unlocknvme0";};
    serviceConfig = {Type="oneshot"; User = "root";
      ExecStart="${pkgs.clevis}/bin/clevis luks unlock -d /dev/nvme0n1p2 -n nvme0crypt";
    };
    wants = [ "network-online.target" ];
    after = [ "network-online.target" ];
    wantedBy = [ "zfs.target" ];
  };

boot.initrd.luks.devices needs to be set, so that device mapper knows the target type crypt (otherwise the error from post 1 is displayed), any recommendations to come around this?

8

Rather than having a dummy device you can just do boot.initrd.luks.forceLuksSupportInInitrd = true; (This option isn’t documented because it’s marked “internal”, but that should probably be changed).

Also I would probably use these dependencies:

wants = ["network-online.target"];
after = ["network-online.target" "cryptsetup-pre.target" "dev-nvme0n1p2.device"];
requires = ["dev-nvme0n1p2.device"];
requiredBy = ["cryptsetup.target"];
# no wantedBy = zfs.target
unitConfig.DefaultDependencies = false;
9

(sorry for editing that like 5 times but I think I’m done now :P)

10

Thanks for that hint, that fixes the need for the dummyDevice

Your listed dependencies do work, but it tries to import the zpool ontop even before setting up network, thats what i tried to come around. No big deal but im trying to get rid of seeing the A start job is running for Import ZFS pool ...-Message until the devices are opened

11

Well, I can’t see how the dependencies you had before would prevent that either. But you can fix that with boot.initrd.services."zfs-import-${poolName}".after = ["cryptsetup.target"]; (which is actually something we’ll probably be adding by default here very soon for better compatibility with scripted initrd)

12

That doesn’t help getting rid of the log message, but i guess i’ll live with it. Thanks for helping :slight_smile:

13

Oh, I forgot to add before = ["cryptsetup.target"]; when I suggested dependencies for your unlock service. That should do it. (Normally you don’t need explicit dependencies for target units that depend on your unit, but DefaultDependencies=false removes this default ordering)

1 Like
14

Not a PR, but my workaround, I want to share. Tested with tang and TPM (if you uncomment tpm2-tools). Assumes systemd-initrd. Feel free to build a PR to the official module with it. The below is WTFPL.

{
  config,
  lib,
  pkgs,
  utils,
  ...
}: {
  boot.initrd.systemd = let
    bins = {
      cryptsetup = "${pkgs.cryptsetup}/bin/cryptsetup";
      curl = "${pkgs.curl}/bin/curl";
    };
  in {
    extraBin = bins // {clevis = "${pkgs.clevis}/bin/clevis";};
    storePaths =
      (builtins.attrValues bins)
      ++ [
        "${pkgs.clevis}"
        "${pkgs.coreutils}"
        "${pkgs.gnused}"
        "${pkgs.gnugrep}"
        "${pkgs.jose}"
        "${pkgs.luksmeta}"
        #"${pkgs.libpwquality}"
        #"${pkgs.tpm2-tools}"
      ];
    services = let
      devicesWithClevis = config.boot.initrd.luks.devices;
    in (lib.mapAttrs'
      (name: value:
        lib.nameValuePair "cryptsetup-clevis-${name}" {
          wantedBy = ["systemd-cryptsetup@${utils.escapeSystemdPath name}.service"];
          before = [
            "initrd-switch-root.target"
            "shutdown.target"
          ];
          wants = ["network-online.target"];
          after = ["systemd-modules-load.service" "network-online.target"];
          partOf = ["systemd-cryptsetup@${utils.escapeSystemdPath name}.service"];
          script = ''
            # wait for device
            retries=0;
            while ! test -b "${value.device}"; do
              retries=$((retries+1));
              echo "Waiting for ${value.device} ($retries/8)"
              sleep 1;
              if test $retries -ge 8; then
                break;
              fi
            done;
            # try unlocking with clevis
            if "${pkgs.clevis}"/bin/clevis luks unlock -d "${value.device}"; then
              systemctl --no-block restart "systemd-cryptsetup@${utils.escapeSystemdPath name}.service";
            fi;
          '';
          conflicts = ["initrd-switch-root.target" "shutdown.target"];
          unitConfig.DefaultDependencies = "no";
          serviceConfig = {
            Type = "oneshot";
            RemainAfterExit = true;
          };
        })
      devicesWithClevis);
  };
}
15

FYI this is probably exploding the size of your initrd by a lot by including stuff like man pages and other unnecessary files from these packages in the initrd. /boot is usually small enough that very large initrd is a real problem. I’d either find the exact files you need or at least a more narrow directory like "${foo}/bin".

You generally shouldn’t write your own wait script. You can just depend on device units, e.g. dev-foo.device in the systemd ordering.

And messing with other systemd units like this is generally not great. It seems like you’d rather replace that unit than have all these weird dependencies and interactions.

16

Thanks for the hints. The size is quite big indeed. I’m used to having bigger boot partitions since before using clevis so I didn’t really notice.

The ugly wait script is something I couldn’t do better. As the service is part of the systemd-cryptsetup unit already it shouldn’t get activated before the device is ready. Yet it is often enough. I’ve seen things like /dev/sda.device beeing ready but /dev/disk/by-uuid/… not yet.

The last part restarts the service to make the passphrase prompt go away. I found no other way of informing systemd the unlock job is done even without someone entering a passphrase. I still wanted to keep the passphrase option so I didn’t replace it.

17

Thanks to the previous discussion I was able to get clevis/tang unlocking working with both scripted initrd and systemd initrd. This doesn’t use the nix clevis module, it’s for disks that have used clevis luks bindto occupy a slot in the LUKS header. I also needed the request to tang to occur over a tagged VLAN, so I extended the configs to do that too.

The systemd config is almost comically long and complex compared to the single command in the scripted initrd, but that’s no surprise and it looks like that’s the future.

Please let me know if you spot any errors or improvements.

Scripted initrd

# Configure LUKS encrypted device
boot.initrd.luks.devices."crypted" = {
  device = "/dev/disk/by-uuid/bfd76164-79ef-4f5e-a491-b7cf78a95e1e";

  # Unlock the device using clevis/tang
  preOpenCommands = ''
    export PATH=$PATH:${pkgs.curl}/bin/:${pkgs.gnused}/bin/
    ${pkgs.clevis}/bin/clevis luks unlock -d /dev/nvme0n1p2 -n crypted
    '';
};

# Undocumented option
#  https://discourse.nixos.org/t/unlocking-luks-devices-at-boot-using-clevis-tang/52512/10
boot.initrd.luks.forceLuksSupportInInitrd = true;

# Enable network in initrd
boot.initrd.network.enable = true;

# Do not use DHCP
boot.initrd.network.udhcpc.enable = false;

# Create a static IP config using the kernel CLI
boot.kernelParams = [
  "ip=192.168.10.99::192.168.10.1:255.255.255.0:myhostname::none"
];

Scripted initd with tagged VLAN

boot.initrd = {
  enable = true;

  # Enable network in initrd
  network.enable = true;

  # Do not use DHCP
  network.udhcpc.enable = false;

  # Allow the use of separate networks for initrd and main system
  network.flushBeforeStage2 = true;

  # Configure LUKS encrypted device
  luks.devices."crypted" = {
    device = "/dev/disk/by-uuid/bfd76164-79ef-4f5e-a491-b7cf78a95e1e";

    # Set up network and unlock the device using clevis/tang
    preOpenCommands = ''
    echo BOOT.INITRD.NETWORK.POSTCOMMANDS START

    echo Activating enp0s31f6...
    ip link set dev enp0s31f6 up

    echo Creating VLAN interface...
    ip link add link enp0s31f6 name management type vlan id 10

    echo Adding IP address to VLAN interface...
    ip addr add 192.168.10.99/24 dev management

    echo Activating VLAN interface...
    ip link set dev management up

    echo Adding default route to VLAN interface...
    ip route add default via 192.168.10.1 dev management

    echo BOOT.INITRD.NETWORK.POSTCOMMANDS END

    echo LUKS DEVICE PREOPENCOMMANDS START

    export PATH=$PATH:${pkgs.curl}/bin/:${pkgs.gnused}/bin/
    ${pkgs.clevis}/bin/clevis luks unlock -d /dev/nvme0n1p2 -n crypted

    echo LUKS DEVICE PREOPENCOMMANDS END
    '';

    # Commands to unlock the device using clevis/tang
    postOpenCommands = ''
    echo LUKS DEVICE POSTOPENCOMMANDS START

    echo Deleting VLAN interface
    ip link delete dev management

    echo LUKS DEVICE POSTOPENCOMMANDS END
    '';
  };

  # Undocumented option
  #  https://discourse.nixos.org/t/unlocking-luks-devices-at-boot-using-clevis-tang/52512/10
  luks.forceLuksSupportInInitrd = true;
};

systemd initrd

# Configure LUKS encrypted device
boot.initrd.luks.devices."crypted" = {
  device = "/dev/disk/by-uuid/bfd76164-79ef-4f5e-a491-b7cf78a95e1e";
};

# Enable network access in initrd
boot.initrd.systemd = {
  enable = true;

  # Begin systemd network configuration
  network = {
    enable = true;

    # Configure ethernet interface
    networks."10-enp0s31f6" = {
      name = "enp0s31f6";
      matchConfig.type = "ether";

      address = [ "192.168.10.99/24" ];
      dns = [ "192.168.10.1" ];
      routes = [ { Gateway = "192.168.10.1"; } ];
      DHCP = "no";
    };

    wait-online.enable = true;
    wait-online.anyInterface = true;
  };
};

# Store utilities needed by clevis in the initrd
boot.initrd.systemd.storePaths = [ "${pkgs.clevis}"
                                   "${pkgs.curl}"
                                   "${pkgs.cryptsetup}"
                                   "${pkgs.luksmeta}"
                                   "${pkgs.gnused}"
                                   "${pkgs.tpm2-tools}"
                                   "${pkgs.jose}"
                                   "${pkgs.libpwquality}"
                                   "${pkgs.coreutils}"
                                   "${pkgs.gnugrep}"
                                   "${pkgs.bash}" ];

# Define a systemd service to run =clevis luks unlock=
boot.initrd.systemd.services.clevis-luks-unlock = {
  enable = true;

  # Put stored utilities on the path of the service
  path = [ "${pkgs.clevis}"
           "${pkgs.curl}"
           "${pkgs.cryptsetup}"
           "${pkgs.luksmeta}"
           "${pkgs.gnused}"
           "${pkgs.tpm2-tools}"
           "${pkgs.jose}"
           "${pkgs.libpwquality}"
           "${pkgs.coreutils}"
           "${pkgs.gnugrep}"
           "${pkgs.bash}" ];

  unitConfig = { Description = "Unlock LUKS encrypted drive using clevis/tang"; };

  serviceConfig = {
    Type="oneshot";
    ExecStart="${pkgs.clevis}/bin/clevis luks unlock -d /dev/nvme0n1p2 -n crypted";
  };

  # Dependencies
  unitConfig.DefaultDependencies = false;
  wants = ["network-online.target"];
  after = [ "cryptsetup-pre.target"
            "dev-nvme0n1p2.device"
            "network-online.target" ];
  requires = [ "dev-nvme0n1p2.device" ];
  requiredBy = [ "cryptsetup.target" ];
  before = [ "cryptsetup.target"
             "systemd-cryptsetup@crypted.service" ];

  # Don't ask for password while this service is running
  unitConfig.Conflicts = "systemd-ask-password-console.path" ;

  # If clevis unlock fails, ask for password to be entered manually
  unitConfig.OnFailure = [ "systemd-ask-password-console.path" ];
};

# Undocumented option
#  https://discourse.nixos.org/t/unlocking-luks-devices-at-boot-using-clevis-tang/52512/10
boot.initrd.luks.forceLuksSupportInInitrd = true;

# Allow the use of separate networks for initrd and main system
boot.initrd.network.flushBeforeStage2 = true;

# Enable emergency access
boot.initrd.systemd.emergencyAccess = true;

systemd initrd with tagged VLAN

# Configure LUKS encrypted device
boot.initrd.luks.devices."crypted" = {
  device = "/dev/disk/by-uuid/bfd76164-79ef-4f5e-a491-b7cf78a95e1e";
};

# Enable network access in initrd
boot.initrd.systemd = {
  enable = true;

  # Begin systemd network configuration
  network = {
    enable = true;

    # Configure VLAN10
    # VLANs should be configured before being used by a network
    netdevs."11-vlan10" = {
      netdevConfig = {
        Kind = "vlan";
        Name = "management";
      };
      vlanConfig.Id = 10;
    };

    # Configure parent ethernet port without IP address
    networks."20-enp0s31f6-untagged" = {
      name = "enp0s31f6";

      # Type is taken from =networkctl list=
      matchConfig.type = "ether";

      # List all VLAN tags to be used
      vlan = [ "management" ];

      # Prevent creation of a link-local address on parent port
      networkConfig = {
        LinkLocalAddressing = "no";
        LLDP = "no";
        EmitLLDP = "no";
        IPv6AcceptRA = "no";
        IPv6SendRA = "no";
      };
    };

    # Configure an interface tagged with VLAN10
    # Analogous to eth0.10 in Debian
    networks."21-enp0s31f6-vlan10" = {
      name = "management";
      matchConfig.type = "vlan";

      address = [ "192.168.10.99/24" ];
      dns = [ "192.168.10.1" ];
      routes = [ { Gateway = "192.168.10.1"; } ];
      DHCP = "no";
    };

    wait-online.enable = true;
    wait-online.anyInterface = true;
  };
};

# Store utilities needed by clevis in the initrd
boot.initrd.systemd.storePaths = [ "${pkgs.clevis}"
                                   "${pkgs.curl}"
                                   "${pkgs.cryptsetup}"
                                   "${pkgs.luksmeta}"
                                   "${pkgs.gnused}"
                                   "${pkgs.tpm2-tools}"
                                   "${pkgs.jose}"
                                   "${pkgs.libpwquality}"
                                   "${pkgs.coreutils}"
                                   "${pkgs.gnugrep}"
                                   "${pkgs.bash}" ];

# Define a systemd service to run =clevis luks unlock=
boot.initrd.systemd.services.clevis-luks-unlock = {
  enable = true;

  # Put stored utilities on the path of the service
  path = [ "${pkgs.clevis}"
           "${pkgs.curl}"
           "${pkgs.cryptsetup}"
           "${pkgs.luksmeta}"
           "${pkgs.gnused}"
           "${pkgs.tpm2-tools}"
           "${pkgs.jose}"
           "${pkgs.libpwquality}"
           "${pkgs.coreutils}"
           "${pkgs.gnugrep}"
           "${pkgs.bash}" ];

  unitConfig = { Description = "Unlock LUKS encrypted drive using clevis/tang"; };

  serviceConfig = {
    Type="oneshot";
    ExecStart="${pkgs.clevis}/bin/clevis luks unlock -d /dev/nvme0n1p2 -n crypted";
  };

  # Dependencies
  unitConfig.DefaultDependencies = false;
  wants = ["network-online.target"];
  after = [ "cryptsetup-pre.target"
            "dev-nvme0n1p2.device"
            "network-online.target" ];
  requires = [ "dev-nvme0n1p2.device" ];
  requiredBy = [ "cryptsetup.target" ];
  before = [ "cryptsetup.target"
             "systemd-cryptsetup@crypted.service" ];

  # Don't ask for password while this service is running
  unitConfig.Conflicts = "systemd-ask-password-console.path" ;

  # If clevis unlock fails, ask for password to be entered manually
  unitConfig.OnFailure = [ "systemd-ask-password-console.path" ];
};

# Undocumented option
#  https://discourse.nixos.org/t/unlocking-luks-devices-at-boot-using-clevis-tang/52512/10
boot.initrd.luks.forceLuksSupportInInitrd = true;

# Allow the use of separate networks for initrd and main system
boot.initrd.network.flushBeforeStage2 = true;

# Enable emergency access
boot.initrd.systemd.emergencyAccess = true;
18

FYI, you really want to be extracting the specific things you need here. As is, you’re including tons of unnecessary files into your initrd, likely including manpages for any of those that don’t use a separate man output, which is making it much much larger than necessary.