Impermanence vs. systemd initrd w/ TPM unlocking

Help
1

Hello,

I have a laptop and a couple of other systems running NixOS that I’ve opted into using BTRFS snapshots for impermanence, Secure Boot using Lanzaboote, SystemD initrd for TPM unlocking of my LUKS partition, and Flakes to manage everything… Despite most of these features being marked as experimental, everything has been working near flawlessly. My system boots up in less than ten seconds without needing to enter a passphrase, with Secure Boot enabled, and system and home configurations defined in flakes that make it super easy to update configurations for multiple systems and users. The “last frontier” so to speak is hibernation and swap. (And FDE.) Here is my repo that drives all of this.

… Then a couple of days ago I realized that I couldn’t use systemd initrd and specify boot.initrd.postDeviceCommands to reset my root subvolume to a blank partition. I got it working without systemd as initrd and didn’t check back on it, so all my system state has been accumulating cruft for a while now. Unfortunately using systemd as initrd is necessary for the TPM unlocking to work, so I have a one-or-the-other situation here.

I haven’t found any indication that anyone has worked around this issue using impermanence via BTRFS or ZFS. Hypothetically it wouldn’t be an issue if using TMPFS, but that’s suboptimal for best performance.

Based on some nixpkgs PRs (167153, 168269) it seems like it should be possible to declare a unit service spec to perform the necessary rollback in stage 1 - has anyone succeeded in doing so, or would know how to modify a NixOS configuration in support of this? I have very little experience when it comes to systemd, and utilizing it via Nix seems very confusing.

1 Like
2

I’m using impermanence with systemd initrd and this service config. I iterated on it in the matrix impermanence chat room with invaluable help from @lilyinstarlight.

  boot.initrd.systemd.services.rollback = {
    description = "Rollback ZFS datasets to a pristine state";
    wantedBy = [
      "initrd.target"
    ]; 
    after = [
      "zfs-import-zroot.service"
    ];
    before = [ 
      "sysroot.mount"
    ];
    path = with pkgs; [
      zfs
    ];
    unitConfig.DefaultDependencies = "no";
    serviceConfig.Type = "oneshot";
    script = ''
      zfs rollback -r zroot/local/root@blank && echo "rollback complete"
    '';
  };
7 Likes
3

That’s exactly what I needed! Thank you so much!

Edit: Here’s where I ended up for the BTRFS version, borrowing from this blog post:


  boot.initrd.systemd.services.rollback = {
    description = "Rollback BTRFS root subvolume to a pristine state";
    wantedBy = [
      "initrd.target"
    ];
    after = [
      # LUKS/TPM process
      "systemd-cryptsetup@enc.service"
    ];
    before = [
      "sysroot.mount"
    ];
    unitConfig.DefaultDependencies = "no";
    serviceConfig.Type = "oneshot";
    script = ''
      mkdir -p /mnt
      # We first mount the btrfs root to /mnt
      # so we can manipulate btrfs subvolumes.
      mount -o subvol=/ /dev/mapper/enc /mnt
      # While we're tempted to just delete /root and create
      # a new snapshot from /root-blank, /root is already
      # populated at this point with a number of subvolumes,
      # which makes `btrfs subvolume delete` fail.
      # So, we remove them first.
      #
      # /root contains subvolumes:
      # - /root/var/lib/portables
      # - /root/var/lib/machines
      #
      # I suspect these are related to systemd-nspawn, but
      # since I don't use it I'm not 100% sure.
      # Anyhow, deleting these subvolumes hasn't resulted
      # in any issues so far, except for fairly
      # benign-looking errors from systemd-tmpfiles.
      btrfs subvolume list -o /mnt/root |
        cut -f9 -d' ' |
        while read subvolume; do
          echo "deleting /$subvolume subvolume..."
          btrfs subvolume delete "/mnt/$subvolume"
        done &&
        echo "deleting /root subvolume..." &&
        btrfs subvolume delete /mnt/root
      echo "restoring blank /root subvolume..."
      btrfs subvolume snapshot /mnt/root-blank /mnt/root
      # Once we're done rolling back to a blank snapshot,
      # we can unmount /mnt and continue on the boot process.
      umount /mnt
    '';
  };
4 Likes