BTRFS -- seeking installation advice

I noticed that if you choose BTRFS for root partition, using the graphical installer, it creates only one subvolume, which is for root. It does not create @home.
Is it recommended to manually create @home, or possibly other subvolumes? Or maybe I should put home on a separate partition?

My main objective in choosing BTRFS is to make snapshots of the home directory, or including the home directory.

To that end, what is a recommended GUI tool for snapshot management? I understand that Timeshift is not exactly tailor-suited to a standard NixOS installation. What about Snapper, or BTRFS-Assistant, or others?

It’s entirely up to how you intend to use the filesystem. The graphical installer isn’t opinionated, it just makes sure to get you a basic btrfs that you can adjust to your needs.

Obviously if you intend to snapshot @home and not everything else you’ll need a separate subvolume for that. Snapshotting /nix is pretty pointless in general too, so you probably want a subvolume for that. That’ll also let you set noacl (which is totally useless on the nix store) for performance and better SSD wear. Nope, I misread those docs.

Personally I like a separate volume for at least /var/log too, due to some PTSD from a poorly configured log file. Then you can set a quota so it doesn’t blow up accidentally (and probably set a ~15G quota for /nix too so you remember to collect garbage) - though be aware of the limitations of using quotas and snapshots.

/var usually gets a subvolume for me, because it’s the dedicated data directory for most system services, and therefore worth snapshotting. I’ll typically make specific subvolumes for directories in /var rather than grabbing the whole thing.

Anything else in / is usually not snapshotted on NixOS because it should only contain configuration that comes from my NixOS config.

That’s just me throwing around my opinions. Gentoo has some good advice for generic distro use too (designed around ext4 and fhs-based systems, so doesn’t fully apply).

Ultimately it comes down to personal preference and experience. btrfs lets you change this stuff very easily without data loss, so don’t worry too much about getting it right immediately - that’s my favorite thing about it compared to older filesystems.

No experience with that, sorry.

6 Likes

Hey, thanks for taking the time to explain that so thoroughly and clearly. I would not have thought of making a subvolune for /var
How to create custom subvolumes in the graphic installer?

3 Likes

So the Wiki on BTRFS installation kind of leaves you hanging over a cliff puzzled what to do
https://nixos.wiki/wiki/Btrfs
because after you created and mounted your subvolumes (so far, so good) it then says “manually add mount options”, but it doesn’t tell you how, and then it tells you to run nixos-install, which will create an installation without a user account or other essentials, because it doesn’t ask any questions. It just does.
Any advice on how to “manually add mount options” and taking it from there on?

The wiki is unofficial and large parts of it, if they were ever of any quality, have bitrotted to hell. There’s an official wiki on the way, we’ll see if it ends up better, but NixOS kind of incentivizes upstream fixes over documenting stuff in a wiki, and the userbase is generally a bit too advanced to write helpful newbie docs, so you end up with this kind of tutorial quite often.

You would use fileSystems.options for that.

Note that btrfs has a subvol option if you want to mount specific subvolumes.

Yes, the non-graphical installer doesn’t hold your hands. The idea is that you have your declarative configuration ready to go and configure everything before you run that command.

I’d suggest you either use the graphical installer, which adds a few sane defaults (though I believe you still need to log in as root initially? Never used it myself), or you follow the manual’s installation docs: NixOS 23.11 manual | Nix & NixOS

Either will give you far more context than the wiki. You’ve got the btrfs part down, at least, so picking up from here with the manual shouldn’t be too hard.

And feel free to fix the wiki! It should at least hand you over to the official guide once the btrfs bit is done.

Okay so what will that look like, if I did like this:

# btrfs subvolume create /mnt/root
# btrfs subvolume create /mnt/home
# btrfs subvolume create /mnt/nix

Just wanted to note, that after you create subvolumes and mount them following the wiki, doing nixos-generate-config --root /mnt will generate basic configuration for mounted filesystems, so after that you can go in and adjust mount options if needed.

So technically, if you follow that wiki to the letter - you should end up with a working configuration

1 Like

Something like what the wiki documents, incidentally:

fileSystems = {
  "/".options = [ "compress=zstd" ];
  "/home".options = [ "compress=zstd" ];
  "/nix".options = [ "compress=zstd" "noatime" ];
  "/swap".options = [ "noatime" ];
};
1 Like

Aha, so you are saying that this step is optional?

# nano /mnt/etc/nixos/configuration.nix # manually add mount options

yes, sorta… nixos-generate-config does detect all the mounts, but I do not remember if it does autodetect all the options used for mounting correctly…
So things like compression you might need to add there into the config.

Is noacl a setting that can be set on the subvolume level? Most options cover the whole filesystem, even if set individually.

1 Like

I think noacl is volume-specific: Subvolumes — BTRFS documentation

I tested it and it’s not.

1 Like

That’s not a problem because one can always tweak the config after installation. I think you resolved my issue.

Let’s say you choose the BTRFS option in the GUI installer, and allow it to give you a single subvolume called “/”

1 What happens if you restore a snapshot? Is it able to remove all those read-only Nix files, and restore the entire system to its former state?

2 If you get tired of such a simplistic setup, can you add subvolumes after installation, simply by defining them in configuration.nix?

Depends on how exactly you plan on restoring the snapshot.

I am not aware of any filesystem-native method of overwriting the btrfs root subvolume (id=5) with another subvolume under it; you’d have to copy each file one-by-one.

Popular snapshotting frameworks assume certain subvolume layouts below the root subvolume for a reason.

Snapshotting /nix is quite useless though and rolling back its state can easily prevent you from booting your system. Don’t do it unless you know exactly what you are doing.

If you get tired of such a simplistic setup, can you add subvolumes after installation, simply by defining them in configuration.nix?

You could technically “add subvolumes by defining them in configuration.nix” in the sense that you can tell systemd-tmpfilesd to create empty subvolumes wherever you like.

You cannot make it automatically switch subvolume layouts for you though. You can tell NixOS to mount any subvolume layout you like but it won’t create or populate that layout for you; that’s up to you.

It is certainly possible to change the subvolume layout of an existing btrfs even while a live system is running on it. I’ve done it a few times. It’s also easy to mess this up though. Have a live distro at hand when attempting this.

Your best option is to manually create the subvolume layout you want and then install NixOS manually. I promise you that both of these are doable by mere mortals and not actually too hard.

2 Likes

Done.
Somehow, hardware-configuration.nix ended up with double entries for some of the subvolumes, and that needed to be fixed before install could proceed, but I got past that hurdle.

1 Like

also something i haven’t yet tried, but looking to switching to in future for automatic partitioning is disco

This should make the process of spinning up a copy of your system much easier

1 Like

Sorry for the bump but I remembered something which invalidates this statement: What gets mounted by default when no subvolume is specified for the mount is the default subvolume, not the root subvol, and it can be changed.

So what you’d do is create a writeable snapshot of the snapshot you wish to restore and make it the default subvolume. If your Nix store mount is not separate, you’d have to copy over the Nix store from root subvol though as its state must match the state in /boot.

1 Like