Feedback on a user guide I created on installing NixOS with ZFS

Help
1

Hey everyone,

I created this guide to be my first post on my non-existent static blog. If you have time, would you mind providing feedback, especially if I have any errors? The formatting will change by the time I post.

Thank you,

Ben

5 Likes
2

Thanks for sharing your guide ! I wanted to install NixOS with encrypted ZFS months ago but I gave up because I had not found a tutorial. Really useful. I will use it for my next install.

I did not know about sgdisk. I really like commands similar to

sgdisk -n 0:0:+1MiB -t 0:ef02 -c 0:grub $DISK

for partitioning.

You could go straight to the main point: the configuration and installation of NixOS on ZFS. Other sections of README.adoc are interesting but do not relate to to the main subject. They could be in other guides IMO. An other idea would be to add a link towards the section of interest, at the top of the README.adoc.

Could you explain zfs option mountpoint=legacy ? And zpool create options ?

There may be two erroneous spaces around equal in SWAP = $DISK-part3

When I installed NixOS on an encrypted luks+ext4 partition, nixos-generate-config wrote all the necessary configuration to ensure that encryption and fs are handled properly: e.g. with a prompt during the boot. I did not need to edit by hand configuration.nix for that. I wonder why it isn’t the case with encrypted ZFS.

You wrote

Before we start editing the configuration files, lets first get our machines networking host id, which is needed by ZFS.

Why does ZFS need the networking host id ?

3

I will expound on the areas you just mentioned, but the hostID is needed (it is in the Nix WIKI on ZFS). It is funny because after I got into the system I cloned my old repo so I had all the programs I wanted, but I forgot to change the hostid (i did that on another computer) and my computer wouldn’t get past the boot menu. I was lucky I was using Nixos and could roll back!

I guarantee if you follow this guide you will have no troubles. Well, on second thought…one thing I am not certain of is the mkswap -U flag. I actually forgot to do this and had to make the swap after the install happened.

4

Host id is needed because ZFS writes it to the pool when it’s imported so that no other machine can mistakenly import the pool until this machine has exported it. This does mean that unclean shutdowns leave your pool in a state that requires other machines to use -f to import the pool. But it helps make sure you haven’t imported the pool in the wrong place, or attempted to import a pool that’s already imported by another system. It’s mostly an enterprise level feature really.

1 Like
5

Ok, thanks. Not 100% convinced by this seatbelt though. IMO it adds unnecessary complexity. Other file-systems – like ext4 – seem to work well enough in production without this feature.

6

Right up until you get bitten by one of the problems it prevents :stuck_out_tongue: I don’t work with this stuff at such an enterprise level though so I have no idea how inevitable that is.

2 Likes
7

I should have written that I just don’t know well enough ZFS and issues with concurrent access to file systems. Use cases – that I cannot imagine clearly yet – must justify this feature.

8

Here are some comments from me on the guide:

  • If you need a wireless connection off the ISO, it is far easier to use Network Manager, especially for a beginner who might be reading a guide like this. Just run nmtui and connect to your wireless network as you would on any other platform.
  • As for ls -l /dev/disk/by-id/ , you can’t guarantee that all disks/partitions will have IDs. Sometimes you will need UUIDs or PARTUUIDs depending on what is available. This will completely break all your instructions where you are relying on things like id-part1 to exist.
  • SDA_ID="$(ls /dev/disk/by-id/ | grep '^[ata]')" is somewhat confusing because it relies on finding a unique string to grep since those prefixes could be shared. This might be simpler and not rely on grep: SDA_ID=$(find -L /dev/disk/by-id -xtype l -samefile /dev/sda 2> /dev/null)
  • You should explain in english what you are doing during the partitioning step. To someone unfamiliar with sgdisk that may just look like gibberish. Especially since you creating a fixed size swap partition that might not be appropriate for everyone.
  • There is a fairly widely used convention to use poolname/ROOT instead of poolname/root. While I personally find it quite annoying it is widely used and probably appropriate to stick with that for a guide.
  • -o ashift=12 - You should probably provide instructions or a link on how to find an appropriate ashift for the drive in question. Otherwise you would be better off not setting it explicitly and letting zfs decide what to use. It certainly shouldn’t be hard set to 12
  • -o altroot="/mnt" - There is no reason to use an alternate root on a zpool which has no zfs mountpoints.
  • While encrypting an entire zpool is an option, it is also an option to only encrypt portions of your zpool which gives you a lot more flexibility without any real downsides as long as you structure it sanely. You might want to at least mention that because once you encrypt a zpool you are pretty much stuck.
  • Your example should include more datasets to illustrate how the structure works at the very least I would add a dataset for /nix so you can set a separate snapshot policy for the nix store. In general, adding more datasets has no downsides but gives you more flexibility so within reason the more the better.
  • snapshots are extremely inexpensive in short-term use so from an illustrative point, it would be good to demonstrate different policies for something like / where snapshots are useful but long-term retention isn’t. As opposed to /home where you might want a much longer retention policy.
  • When you enable autosnapshots you should also set a default policy for frequency and retention
  • You could mention that if they have an SSD is important to set services.zfs.trim.enable = true;
  • It is often needed to set boot.zfs.devNodes = "/dev/disk/by-id";
4 Likes
9

zfs has 3 mount options:

  • none which means the dataset has no mountpoint
  • A zfs mountpoint like /home which says that zfs should handle mounting the filesystem
  • legacy which means that zfs will make it available to be mounted by the filesystem using mount

For nixos, your life will be much easier if you use legacy or none for everything.

If you are looking for simplicity, zfs is not the right tool. :wink:

zfs is far more complicated(and functional) than a basic filesystem like ext4.

2 Likes
10

This is an amazing response. I really didn’t think I would receive feedback this good. I will update the document with these points.

3 Likes