Disko and ZFS, emergency mode during boot

Help
1

Greetings,

I’m fairly new to NixOS and I’m trying to setup a NAS for use at home. During the installation of a new server (NixoS 24.11) I used Disko to setup a ZFS pool, here is the relevant section:

zpool = {
  ironwolf = {
    type = "zpool";
    mode = {
      topology = {
        type = "topology";
        vdev = [{
          mode = "mirror";
          members = ["sda" "sdb"];
        }];
      };
    };
    mountpoint = "/mnt/ironwolf";
    datasets = {
      media = {
        type = "zfs_fs";
        #options.mountpoint = "legacy";
        mountpoint = "/mnt/ironwolf/media";
      };
      appdata = {
        type = "zfs_fs";
        #options.mountpoint = "legacy";
        mountpoint = "/mnt/ironwolf/appdata";
      };
    };
  };
};

When the server boots it successfully imports the pool, but fails to mount the datasets, here is the output:

[ OK ] Finished Import ZFS pool "ironwolf".
[ OK ] Reached target ZFS pool import target.
          Mounting /mnt/ironwolf...
          Starting Mount ZFS filesystems...
[FAILED] Failed to mount /mnt/ironwolf.
See 'systemctl status mnt-ironwolf.mount' for details.
[DEPEND] Dependency failed for Local File Systems.
[DEPEND] Dependency failed for /mnt/ironwolf/appdata
[DEPEND] Dependency failed for /mnt/ironwolf/media

Then it goes into emergency mode. If I press Ctrl-D to continue, the system boots successfully and my ZFS pool is online and healthy with all datasets mounted.

After reading some similar posts here, I’ve found both ZFS and Systemd are attempting to mount the datasets, but I don’t know which one is failing or how to tell it to use just one. As you can see from the Disko config I’ve tried using legacy mountpoints but this doesn’t seem to work. I’ve also tried adding boot.zfs.extraPools = [ "ironwolf" ]; to the main configuration at various points and there are no filesystems defined in the hardware configuration, but I always end up in emergency mode.

Any help here is much appreciated, thank you in advance :slight_smile:

2

I’ve managed to get the server to boot without going into emergency mode by telling systemd to not mount the zfs datasets. I added systemd.services.zfs-mount.enable = false; to my configuration and now it’s working.

Does anyone know if doing this is not recommended and if so, how do I get systemd to mount the datasets correctly?

3

I’m hitting a very similar issue - did you find a solution?

4

It doesn’t matter which one is failing (though in the log you shared it’s systemd). It’s a race condition as to what’s going to happen. If you’ve got non-legacy mountpoints on those datasets, they should not be in your fileSystems Nix config, and it looks like they are. I don’t know if that’s because you have them in your hardware-configuration.nix or if it’s because disko is creating those entries (I don’t know much about disko), but for non-initrd, non-legacy datasets, they shouldn’t be in fileSystems at all.

By disabling zfs-mount.service, btw, you’re forgoing ZFS’s automatic management of mount options, and you’re only getting basic mount options. That’s part of why zfs-mount.service is preferable to having non-legacy datasets in fileSystems

5

Hi @kernald,

After a lot of experimentation I did end up with a configuration I’m happy with. Here is the relevant part of my disko.nix:

zpool = {
      ironwolf = {
        type = "zpool";
        rootFsOptions = {
          compression = "lz4";
          acltype = "posixacl";
          xattr = "sa";
          atime = "off";
          canmount = "on";
        };
        mode = {
          topology = {
            type = "topology";
            vdev = [{
              mode = "mirror";
              members = ["sda" "sdb"];
            }];
          };
        };
        datasets = {
          media = {
            type = "zfs_fs";
            options = {
              compression = "lz4";
              canmount = "on";
              mountpoint = "/mnt/ironwolf/media";
            };
          };
          appdata = {
            type = "zfs_fs";
            options = {
              compression = "lz4";
              canmount = "on";
              mountpoint = "/mnt/ironwolf/appdata";
            };
          };
        };
      };
    };

I also have a zfs.nix included in the configuration as follows and it no longer includes the systemd.services.zfs-mount.enable = false; line I mentioned above:

{ pkgs, ... }: {
  boot = {
    supportedFilesystems = [ "zfs" ];

    zfs = {
      forceImportRoot = true;
      forceImportAll = true;
      devNodes = "/dev/disk/by-id";
      extraPools = [ "ironwolf" ];
    };
  };

  services.zfs = {
    autoScrub.enable = true;
    trim.enable = true;
  };

  # Create and manage zpool.cache
  systemd.services.zfs-setup = {
    description = "Setup ZFS pools";
    serviceConfig = {
      Type = "oneshot";
      RemainAfterExit = true;
    };
    script = ''
      mkdir -p /etc/zfs
      ${pkgs.zfs}/bin/zpool set cachefile=/etc/zfs/zpool.cache ironwolf
    '';
    wantedBy = [ "zfs.target" ];
  };
}

I hope this helps :slight_smile:

6

That makes sense. Another option with Disko, using legacy mountpoints, is something like this:

datasets = {
  root = {
    type = "zfs_fs";
    mountpoint = "/";
    options.mountpoint = "legacy";
  };
};

The issue stems as pointed out by ElvishJerricco above from zfs and systemd both trying to mount the datasets, if not enabling legacy mountpoints.

1 Like
7

I am fairly sure this should not be necessary

8

Oh I didn’t realize there was a difference between mountpoint and options.mountpoint in that disko ZFS stuff. Yea, you probably either want options.mountpoint = "/a/path"; or mountpoint = "/a/path"; + options.mountpoint = "legacy";.

9

Yo, I have exactly the same issue.
disko-config.nix:

    zpool = {
      zroot = {
        type = "zpool";
        rootFsOptions = {
          acltype = "posixacl";
          atime = "off";
          compression = "zstd";
          mountpoint = "none";
          xattr = "sa";
        };
        options.ashift = "12";

        datasets = {
          "root" = {
            type = "zfs_fs";
            options = {
              mountpoint = "/";
              canmount = "on";
              "com.sun:auto-snapshot" = "false";
            };
          };
          "home" = {
            type = "zfs_fs";
            options = {
              mountpoint = "/home";
              canmount = "on";
              # Used by services.zfs.autoSnapshot options.
              "com.sun:auto-snapshot" = "true";
            };
          };
        };
      };
    };
  };
}

Also have a simmiliar zfs.nix

Unfortunately now there is an error:

Failed assertions:
- The ‘fileSystems’ option does not specify your root file system.

So I guess I must use legacy mount for root directory right?