So my final goal was to create a NAS. with nixos installed on SSD and then 3 hdd drives for the zfs pool. I have littler experiences with zfs but after playing with it a bit on ubuntu I got some rudimentary understanding how it works.
So my initial idea was to install the OS on a fast SSD, which can be much smaller then the 3 HDD. drives and so I thought to install it on ext4 file system and create zfs pool for the other 3 HDD which will be used to store data with some redundancy.
Right now I am using disko for all of my machines, and what I like about disko is that it can be used during installation with nixosanywhere to format the hard drives and then to configure them, so no need to configure mount points with flileSystems.
for example this is a config I use for my VM single drive vda
{
disko.devices = {
disk = {
vda = {
device = "/dev/vda";
type = "disk";
content = {
type = "gpt";
partitions = {
ESP = {
type = "EF00";
size = "500M";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
};
};
root = {
size = "100%";
content = {
type = "filesystem";
format = "ext4";
mountpoint = "/";
};
};
};
};
};
};
};
}
So based on this understanding I tried the example provided by disko for zfs with ext4
# systemd will mount an ext4 filesystem at / and zfs will mount the dataset underneath it
{
disko.devices = {
disk = {
disk1 = {
type = "disk";
device = "/dev/vda";
content = {
type = "gpt";
partitions = {
ESP = {
size = "500M";
type = "EF00";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
mountOptions = [ "umask=0077" ];
};
};
primary = {
size = "100%";
content = {
type = "filesystem";
format = "ext4";
mountpoint = "/";
};
};
};
};
};
disk2 = {
type = "disk";
device = "/dev/vdb";
content = {
type = "zfs";
pool = "zroot";
};
};
};
zpool = {
zroot = {
type = "zpool";
datasets = {
"root" = {
type = "zfs_fs";
options.mountpoint = "none";
};
"root/zfs_fs" = {
type = "zfs_fs";
mountpoint = "/zfs_fs";
options."com.sun:auto-snapshot" = "true";
};
};
};
};
};
}
Which was exactly what I am aiming for(SSD for os, and HDD for zfs pool).
Installation with nixosanywhere goes just fine but system fails to boot, it gets stuck on importing zfs pool(zppol).
I also tried to go full zfs, I took disko example and trimmed it to bare bones.
{
disko.devices = {
disk = {
x = {
type = "disk";
device = "/dev/vda";
content = {
type = "gpt";
partitions = {
ESP = {
size = "64M";
type = "EF00";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
mountOptions = [ "umask=0077" ];
};
};
zfs = {
size = "100%";
content = {
type = "zfs";
pool = "zroot";
};
};
};
};
};
y = {
type = "disk";
device = "/dev/vdb";
content = {
type = "gpt";
partitions = {
zfs = {
size = "100%";
content = {
type = "zfs";
pool = "zroot";
};
};
};
};
};
};
zpool = {
zroot = {
type = "zpool";
mode = "mirror";
# Workaround: cannot import 'zroot': I/O error in disko tests
options.cachefile = "none";
rootFsOptions = {
compression = "zstd";
"com.sun:auto-snapshot" = "false";
};
mountpoint = "/";
postCreateHook = "zfs list -t snapshot -H -o name | grep -E '^zroot@blank$' || zfs snapshot zroot@blank";
datasets = {
zfs_fs = {
type = "zfs_fs";
mountpoint = "/zfs_fs";
options."com.sun:auto-snapshot" = "true";
};
};
};
};
};
}
But nothing works, At worst I get stuck at the failure to import with the option to press Enter and go into emergency mode(which fails) and at best I was able to boot into the OS after zpool failure but the pool was not mounted(was able to mount it manually).
A few times I got to the point where it looks like the import worked but importing the data sets failed.
I did some reading and it is my understanding that systemd and zfs might go into race condition so I also tried to use legacy mode for zfs but it did not work for me.
As I mentioned before, disko should handle all the mount config, so I started with no additional boot configs other then what I had with ext4
{
boot.loader.systemd-boot.enable = true;
boot.loader.efi.canTouchEfiVariables = true;
}
then tried to add
boot.supportedFilesystems = [ "zfs" ];
#boot.zfs.enabled = true;
boot.zfs.extraPools = [ "zroot" ]; # Ensure NixOS imports this pool at boot
networking.hostId = "3F8C6B19"; # Replace with your unique host ID
services.zfs.autoSnapshot.enable = true; # Enables automatic snapshots
services.zfs.trim.enable = true; # Enables TRIM for SSDs
with no success.
I am looking for an input, or better yet a working example.
Also feel free to comment on my NAS design.
the issue was fixed by explicitly setting.