Hi, I am trying NixOS and one of the things I like is the clutter-free booting with just an EFI partition plus a non-crippled ZFS pool. However I am struggling to get it completely “clean”.
The wiki (https://nixos.wiki/wiki/ZFS) seems to imply that one can drop the legacy mountpoints by adding boot.zfs.extraPools. When I do this, and remove fileSystems."*"entries in hardware-configuration.nix (except for /boot of course) then (i) either the system does not boot (in case I remove e.g. /nix), or (ii) an assertion fails during nixos-rebuild (“The fileSystems option does not specify your root file system”).
I am sure the ZFS datasets can be made to load during boot just fine. E.g. according to other answers to https://discourse.nixos.org/t/zfs-systemd-boot/29956 one could import all ZFS datasets including root using boot.initrd.postDeviceCommands.
However, how do I get past this failing assertion?
I am just a newbie regarding both Nixos and zfs, but I try to give you a couple of ideas (I hope the experts will give you better help).
First, I would try to get a working setup with legacy mounts. If it works, then at least we have a working starting point.
Second, for what I see, all the examples I was able to find uses the legacy mountpoints, included the official guide to zfs on open-zfs site. So this make me wander whether new mount points are actually supported, or if there are reasons that prevent people to use them.
About the wiki, have you tested the parameters reported in this section?
Here I link the repo in which I am sharing the exploration I am doing regarding Nixos, so you can have a look to the zfs part, maybe it could be useful for you:
Please note that it is a work in progress. Also, I was able to mount by using the boot.zfs.forceImportRoot = true; option, without it I was not able find a set of working configurations. Maybe you just need it as well.
Btw, I am currently working on other topics, but as soon as possible i will turn back to work on zfs, so in case you are able to find a solution, please share.
I’ve not tried this (still learning Nixos so not tried anything advanced) but here are some thoughts…
You may need boot.zfs.extraPools = [ “zpool_name” ]; instead of the legacy items (so this doesn’t really help your clean-up).
The pool needs to be created with canmount=on - the OpenZFS wiki has the boot and root with canmount=off, so if you followed this guide I don’t think it will work.
My current setup is not using legacy mountpoints and is booting correctly. The “issue” I am having is related to the “cleanness” of the setup: even though ZFS could be managing all mounts, that is not working. Instead, I still have to enumerate all ZFS mountpoints in hardware-configuration.nix. So this kind of defeats the purpose.
I will try to reformulate the question this afternoon, as experimentation has lead me into a rabbit hole of initrd details with /mnt-root issues. In the meantime, if someone could point to a clear explanation of NixOS boot phases and all? I am plowing through the code now
These have to be mounted by stage 1. Datasets with non-legacy mountpoints are mounted in stage 2 by a service called zfs-mount.service. You can keep the non-legacy mountpoints on these stage 1 file systems, but there’s not really any benefit to it because you do need the fileSystems entries, and you have to add zfsutil to their options if you do.
How would systemd find the units, if you didn’t mount /etc yet? Which again requires /nix/store to be mounted for the cascade of links to follow.
So to be able to start that service, you need at least /, /nix, /nix/store and /etc mounted the “legacy” way.
Just do yourself a favor, and keep the stage 1 entries, and do the rest in ZFS mounts. Your config will still be a lot cleaner than mine, where I have to mount every LV possible, as LVM does not have an equivalent of ZFS mounts.
During stage 1 zfsutils are loaded, so mounting all datasets at their respective mount points should be no problem. The thing is, I have no idea of the specific role of /mnt-root yet, but that should be solvable.
So what might be possible: during stage 1, mount all ZFS datasets, do a bind mount to /mnt-root (again, I don’t yet understand what role this plays so maybe something else is better), then, at stage 2 hand everything over to zfs-mount.service (which may come to the conclusion that all mounts are already established).
So while I have other fish to fry, I still believe a ZFS-native mount process during boot is possible. NixOS just doesn’t support it yet.
