Help, I want to reinstall NixOS with btrfs subvolumes

Help
1

Hello everyone. After lots of tinkering I realised that my current system did not contain any btrfs subvolumes and I tried to create them manually but it made a mess in my file system. Therefore, I decided to wipe the drive and start fresh.

I have read the wikis, I tried to puzzle all the bits and pieces of information about this topic here and there. I am unsure what I have to do so I would like to have some help reinstalling my system.

Since the Gnome GUI installer doesn’t create any btrfs subvolumes by default, I will have to manually install the system through the terminal.

I have got a 2TB nvme ssd so I planned to do:
500MB for boot (FAT32 or VFAT [What is the point of VFAT and does it work as boot?)
1.5TB for root with all the directories mounted to their relative subvolumes from the get go.

> @
> @home
>   @.cache
> @var
>  @var/cache
>  @var/log
>  @var/lib
> @tmp
> @swap

This is my plan but I do not know how to create them and mount them because I am so confused.
I also wonder why /boot has to be FAT32 format, can it be BTRFS as well?

After I got the parititioning done, I will need to install the system.
If you have been following my posts from before, I live in China and I had trouble accessing nixOS’ official server, any server file downloads will slow to basically unusable and might corrupt the file, in which I suspect that might be the reason my system will randomly crash and frozen. so I need Chinese mirrors for the channel and binary cache. I got them working in my current system but I do not know how to use them while installing the system.

Context:
https://discourse.nixos.org/t/help-i-broke-my-nix-channel/58309
https://discourse.nixos.org/t/help-nixos-kde-plasma-6-kept-crashing-and-freezing-even-in-tty/58425

I also got my current configuration.nix that I am pretty happy with because it contains fixes for a lot of issues. I would like to apply them from the get go. I wonder if there is any " nixos-rebuild switch" equivalent for system install?

Thank you for reading my post.

BTW, I understand that I need to create a swap subvolume in order to isolate it from the other directories so that snapshots can be used. But I am confused after reading the wiki.

> Swap file
> 
> Optionally, create a separate subvolume for the swap file. Be sure to regenerate your hardware-configuration.nix if you choose to do this.
> 
> # mkdir -p /mnt
> # mount /dev/sdXY /mnt
> # btrfs subvolume create /mnt/swap
> # umount /mnt
> # mkdir /swap
> # mount -o noatime,subvol=swap /dev/sdXY /swap
> 
> Then, create the swap file and adjust its size as desired:
> 
> # btrfs filesystem mkswapfile --size 8g --uuid clear /swap/swapfile
> 
> Finally, add the swap file to your configuration and nixos-rebuild switch:
> 
> swapDevices = [ { device = "/swap/swapfile"; } ];

It did not use btrfs to create any subvolume but it asks me to mount it?
I am sorry I just confused.

2

Alright, nobody responded.

I will do some tinkering as long as I don’t mess up my Windows drive.

3

Hm, well, kind of a big question, lemme try:

This is part of the UEFI spec. Your motherboard needs to be able to read the partition, and the spec dictates that you should format it FAT so that there’s a standard that makes this easy. Microsoft basically declared the standard, so it shouldn’t be surprising that their filesystem was chosen.

That said, the spec doesn’t prohibit other filesystems. Some motherboards may support other filesystems. This is very rare, so it’s best to use FAT anyway, but if you have e.g. an Apple computer they add support for their pet filesystem.

I sincerely doubt anything supports btrfs though, that’s way too complex to just write a UEFI driver for because you want to.

It’s also kinda pointless to use anything complex here, the boot partition doesn’t contain anything valuable (at least on NixOS), so you won’t need snapshots, and it’s supposed to be tiny, especially for modern drive sizes, so compression and co. don’t do much for you. Journalling might make boot failures subtly less likely, but you don’t write to /boot often so it’s rare to see issues, and restoring even if it is corrupted is super easy since nothing of value is lost with a reformat.

AIUI VFAT is a hack to encode longer filenames than FAT usually allows that is compatible with drivers that don’t support it, but results in some name truncating in those cases. Early filesystems were weird…

I was curious, so I looked it up, and looks like the spec doesn’t mandate VFAT support in any obvious way. I’d guess this means (since VFAT is backwards compatible) that VFAT would work, but you risk files overriding each other only at boot and therefore not booting what you expected.

I.e., I’d recommend FAT32 without VFAT, as that will refuse to create systems that have boot weirdness. But in practice it’s unlikely to matter.

There’s a command there that creates a subvolume, and afterwards you create a so-called swap file. That is, a normal file inside that subvolume that is formatted correctly to be used as swap. You can mount stuff like that, yeah.

I’m not sure what the confusion is here, that should work.

With that said, there are a bunch of limitations to btrfs-based swapfile, and it doesn’t really give you many benefits over traditional swap. The most notable issue being that hibernation is way harder to do, which is basically the primary purpose of swap on a desktop system.

If you have a drive that large, I would seriously suggest just making a separate swap partition and formatting it with mkswap instead. It’ll save you confusion later on, when something that should work doesn’t, and nobody can help you because your system is weird.

Well, frankly this might be a bit of a problem long-term, if you want to use btrfs you’ll have to use the command line to access any of its features anyway.

I’d frankly recommend doing at least one installation just following the manual install guide using ext4. That’ll get you used to the basics of partitioning by hand.

If you follow that guide, to produce a btrfs-based system, instead of this (assuming you follow my advice about swap):

parted /dev/sda -- mkpart root ext4 512MB -8GB

You would run:

parted /dev/sda -- mkpart root btrfs 512MB -8GB

Then in the formatting section instead of this:

mkfs.ext4 -L nixos /dev/sda1

You would run:

mkfs.btrfs -L nixos /dev/sda1

Since you only have one drive, you shouldn’t do anything with the other options. mkfs.btrfs will pick good defaults for your use case.

After that, you need to create your subvolumes. The wiki tells you how to do that quite well, for sub-subvolumes you just add a / to the path e.g.:

btrfs subvolume create /mnt/root/var/cache

That said, I would suggest just sticking to the wiki’s suggestion for now, so you have a system on which you can learn how to use btrfs. Btrfs lets you easily create new subvolumes later, you’re not locked-in like with partitioning schemes, so there’s no point in getting fancy before you have a running system.

And also, yeah, maybe stick to ext4 until you have a few manual installations under your belt and really understand how this works.

And finally… You’ll need to put your substituters in nix’ configuration file, so with the installer shell (use e.g. nano) write to ~/.config/nix/nix.conf:

substituters = https://mirrors.tuna.tsinghua.edu.cn/nix-channels/store

Alternatively, add to the nixos-install command the flag --option substituters "https://mirrors.tuna.tsinghua.edu.cn/nix-channels/store".

Hope that helps! You’re certainly exploring here :wink:

Tagging @Shiroxia since you have two accounts.

4

Sorry for the confusion,
I would like to ask about why it umount /mnt and mkdir /swap.

Since it already created a subvolume under mnt/swap, why did it made another swap folder under /?
Can I just create a swap file in /mnt/swap?

Furthermore, in the wiki’s tutorial

mkfs.fat -F 32 /dev/sdX1

mkfs.btrfs /dev/sdX2

Neither command specified the size of each partition, I am not sure if they are correct or not.

5

Ah, heh, good question. My guess is this is because the nixos-generate script isn’t smart enough to figure out where to mount a swapfile, but AIUI it wouldn’t be on the actual drive you install to then. No idea, maybe try it and see what it does?

The size is determined by parted, not the filesystem creation command.

6

Thanks, then it means that the wiki page is incomplete because the parted command is nowhere to be found.

7

Yes, I think it assumes you’ve followed the manual install guide before and roughly know what you’re doing. This has the parted commands you need: NixOS Manual, though of course, replace ext4 with btrfs for btrfs.

8

I just found a much better way of crrating swap.

https://discourse.nixos.org/t/how-to-add-a-swap-after-nixos-installation/41742

No need for partition or subvolume, it will just create a swap file under root at the top level.

9

Whelp, nevermind, double checked the code, the module is more clever than I thought: nixpkgs/nixos/modules/config/swap.nix at 1dab772dd4a68a7bba5d9460685547ff8e17d899 · NixOS/nixpkgs · GitHub

You probably still want that swapfile in a different subvolume, though, since btrfs will still not allow a whole bunch of operations on a subvolume containing that file. Also test if hibernation works early, btrfs has some special requirements if you want to do that from a swapfile.

And in case anyone else comes across this, be careful when creating swapfiles on btrfs manually; it needs special handling.