Install of zfs and impermanence causes failure to find nixos closure

Making a new server to host stuff on, and when I attempted to use impermanence with a zfs rollback I consistently get the error “failed to find the nixos closure”. Not sure what issue I’m running into here on account of the only error message I’m seeing is that some file init in /sysroot (or the nix store?) doesn’t exist. If I don’t use impermanence and the rollback everything works fine. Another clue I found but have no idea what to do with is that the persist, run, and var folders will be left behind after I boot back into the rescue usb. Any idea what I’m missing thats causing these issues?

configuration.nix

{ config, lib, pkgs, ... }:

{
  imports = [ ./hardware-configuration.nix ];

  nixpkgs.config.allowUnfree = true;
  nix = {
    package = pkgs.lixPackageSets.latest.lix;
    settings = {
      experimental-features = [ "nix-command" "flakes" "pipe-operator" ];
      auto-optimise-store = true;
    };
    gc = {
      automatic = true;
      dates = "weekly";
      options = "--delete-older-than 7d";
    }; 
  };
  boot = {
    kernelParams = [ "nohibernate" "nomodeset" ];
    supportedFilesystems = [ "zfs" "exfat" "fat32" ];
    zfs.forceImportRoot = false;
    kernelModules = [ "zfs" ];
    loader = {
      systemd-boot.enable = true;
      efi.canTouchEfiVariables = true;
    };
    initrd = {
      supportedFilesystems = [ "zfs" "exfat" "fat32" ];
      kernelModules = [ "zfs" ];
      systemd = {
        emergencyAccess = true;
        extraBin = {
          "zfs" = "${pkgs.zfs}/sbin/zfs";
        };
        enable = true;
        services.initrd-rollback-root = {
          after = [ "zfs-import-zpool.service" "initrd-root-device.target" "local-fs-pre.target" ];
          requiredBy = [ "initrd.target" ];
          before = [ "sysroot.mount" ];
          requires = [ "initrd-root-device.target" ];
          unitConfig.DefaultDependencies = false;
          serviceConfig = {
            Type = "oneshot";
            StandardOutput = "journal+console";
            StandardError = "journal+console";
          };
          script = ''
            zfs rollback -r zpool/local/root@blank
          '';
        };
      };
    };
  };
  hardware = {
    enableRedistributableFirmware = true;
    cpu.intel.updateMicrocode = true;
  };
  networking = {
    hostName = "aquaProdServer";
    hostId = "0dc92367";
  };
  systemd = {
    sleep.settings.Sleep = {
      AllowSuspend = "no";
      AllowHibernation = "no";
      AllowHybridSleep = "no";
      AllowSuspendThenHibernate = "no";
    };
    network.networks."10-wan" = {
      matchConfig.name = "enp1s0f3";
      address = [
        "192.168.1.222/24"
      ];
      routes = [
        { Gateway = "192.168.1.1"; }
      ];
      linkConfig.RequiredForOnline = "routable";
    };
  };
  users = {
    mutableUsers = false;
    users = {
      aqua = {
        enable = true;
        isNormalUser = true;
        extraGroups = ["wheel"];
        hashedPassword = "$y$j9T$njh/t7.//7o3W7vwQPf27/$Ez.x/UfGe8OjC3kYSID9BnHEJjIni2CjxoOLZg3IP84";
      };
    };
  };

  services = {
    getty.autologinUser = "aqua";
    zfs = {
      autoScrub.enable = true;
      trim.enable = true;
    };
    xserver.enable = false;
  };

  fileSystems."/" = {
    device = "zpool/local/root";
    fsType = "zfs";
  };

  fileSystems."/persist" = {
    device = "zpool/safe/persist";
    fsType = "zfs";
    neededForBoot = true;
  };
  
  environment = {
    persistence."/persist" = {
      enable = true;
      hideMounts = true;
      directories = [
        "/etc/nixos"
        "/var/lib/nixos"
      ];
    };
    systemPackages = with pkgs; [
      vim
    ];
  };

  system.stateVersion = "26.05";
}

What do you mean by “left behind” please?

(BTW, if you happen to have plenty of RAM then you might consider using a tmpfs as root instead of a ZFS dataset, since you’re rolling it back every boot.)

Am I missing something or did you just not persist /nix? If you’re just rolling back /nix every boot then yea you’re going to lose all the parts of your nix store that are a part of your new generations

1 Like

I do not have an abudance of ram unfortunately. When I say left behind I mean the folders are still there when I mount the zpool back to /mnt

This is probably the issue, let me add the folder to the impermanence config and see if it fixes it

So if I add /nix to the list of directories persisted by impermanence, the nix folder created inside of /persist is actually empty after I do nixos-install. I’m guessing I just have to stop being lazy and add /nix into its own zpool/safe/<some_name>

If that is the case I feel like this should probably be added to the impermanence github readme as a warning somewhere

IIUC impermanence doesn’t do any mounting for you during nixos-install, only during boot. So if you set up a persistent directory in the nixos config, that’s just not automatically set up during nixos-install; you have to manually do all the same things before nixos-install AFAIK.

Manually copying the files into /persist afterwards and then rebooting does appear to fix the issue. I will try to remember in the morning to make a github issue about adding the fact it doesn’t do that in nixos-install to the readme, but for now I’m going to go to sleep. Thank you for your help with this, system is finally working