NixOS btrfs won't boot: invalid mount argument

When I try to install and boot NixOS, I get the following error:

<<< NixOS Stage 1 >>>

loading module btrfs...
loading module dm_mod...
running udev...
Starting systemd-udevd version 254.3
Passphrase for /dev/disk/by-id/nvme-WD_BLACK_SN770_500GB224448800320-part2:
Verifying passphrase for /dev/disk/by-id/nvme-WD_BLACK_SN770_500GB224448800320-part2... - successs
starting device mapper and LVM...
Scanning for Btrfs filesystems
registered: /dev/mapper/crypted
mounting /dev/disk/by-id/nvme-WD_BLACK_SN770_500GB224448800320-part2 on /...
mount: mounting /dev/disk/by-id/nvme-WD_BLACK_SN770_500GB224448800320-part2 on /mnt-root/ failed: invalid argument

An error occurred in stage 1 of the boot process, which must mount the
root filesystem on `/mnt-root' and then start stage 2.  Press one 
of the following keys:
  r) to reboot immediately
  *) to ignore the error and continue

I think that invalid mount argument must be due to something wrong in the NixOS generated symlinked /etc/fstab, which looks like this:

# <file system>                                             <mount point> <type> <options>                                        <dump> <pass>
/dev/disk/by-id/nvme-WD_BLACK_SN770_500GB224448800320-part2 /             btrfs  x-initrd.mount,subvol=root,compress=zstd,noatime 0      0
# other fs'es here ...

Note that on a live USB, I can mount the system by doing:

$ cryptsetup open /dev/disk/by-id/nvme-WD_BLACK_SN770_500GB224448800320-part2 crypted --key-file $key_file
$ mount /dev/mapper/crypted /mnt-root/ -o compress=zstd -o noatime -o subvol=root -o X-mount.mkdir
$ ls /mnt-root
etc bin var tmp srv other_dirs...

My hardware-configuration.nix looks like:

fileSystems = {
  "/" = {
    device = "/dev/disk/by-id/nvme-WD_BLACK_SN770_500GB224448800320-part2";
    fsType = "btrfs";
    options = [ "subvol=root" "compress=zstd" "noatime" ];
  };
  # other fs'es here...
}
boot = {
  initrd.luks.devices.crypted.device = "/dev/disk/by-id/nvme-WD_BLACK_SN770_500GB224448800320-part2";
  # other options here ...
};
# other options here ...

which loosely follows https://mt-caret.github.io/blog/posts/2020-06-29-optin-state.html.

So the output that you provide here looks like ${disk} should have been replaced with something but actually hasn’t. But maybe I am just confused.

Other than that, what would ${disk}-part2 refer to? On my system disks are called something like nvme0n1p2 for nvme drive 0, namespace 1, partition 2. Now I mount them by uuid, but since you want to do it in this way, there should be something akin to my designation showing up on your end.

Because they were potentially confusing, I removed the references to ${disk}, which were originally just to save typing when posting this question (not actually in the output of initrd). When using by-id, the disk is /dev/disk/by-id/nvme-... and the second GPT partition (where BTRFS is installed) is /dev/disk/by-id/nvme-...-part2. Does that make sense?

Perhaps I should switch to /dev/disk/by-uuid/...

So on my installation the naming scheme for disks and paritions has no -part2 in it. If you want to mount the second partition on the first nvme disk in your system it most likely will be like this: nvme0n1p2.

Can you maybe provide the output of ls -la /dev/disk/by-id on your system?

Edit: I might have been a bit confused. Still the output would be helpful I guess :smiley:

(from live USB)

$ ls -l /dev/disk/by-id
lrwxrwxrwx 1 root root 13 Dec  2 18:02 nvme-WDBLACK_SN770_500GB_224448800320 -> ../../nvme0n1
lrwxrwxrwx 1 root root 15 Dec  2 18:02 nvme-WDBLACK_SN770_500GB_224448800320-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 15 Dec  2 18:02 nvme-WDBLACK_SN770_500GB_224448800320-part2 -> ../../nvme0n1p2
other partitions, other disks ...

Anyway, I switched /dev/disk/by-id/nvme-WDBLACK_SN770_500GB_224448800320-part2 to /dev/disk/by-uuid/big-long-uuid, which are both themselves symlinks to /dev/nvme0n1p2. Still same error.

Yes, that question has gotten more interesting. I am not really sure right now what the error might be here. Other than not using uuids your config does not look much different from mine. Maybe someone else has another idea.

Are there any tricks I need to get btrfs support enabled in Linux and in GRUB?

btrfs should work without any extra tricks, but i’ve never tried luks, therefore i suspect it’s something to do with it. For example - it’s trying to mount device, but is it actually trying to mount the mapped decrypted device?
i wonder also that doc say config for luks goes in boot.initrd.luks, not in boot.luks - can it affect?

Yes, I noticed that too. I have this line in my config:

boot.initrd.luks.devices."nixenc".device = "/dev/disk/by-uuid/some-uuid-here";

Where I think nixenc is the name of the LUKS container. But it has been a while since I set this up.

But what is interesting maybe is that the uuid of my nixenc device is different from the uuid of the device from which the filesystems are mounted.

I should also say I set this up manually any time I need to. And I use systemd-boot, not grub.

I just typed that in incorrectly in this forum; I’ve edited the post to reflect my actual hardware-configuration.nix.

Can you give that part of your config and ls -l /dev/disk/by-uuid? I wonder why the nixenc device is different from the btrfs device…

hi! So i’ve tried to repeat the basic steps of setting up luks on a VM, and indeed, uuids for actual disc and whatever luks decrypts are different, here’s my example:
ll /dev/disk/by-uuid


(vda - is the virtual disc that is being encrypted, and the last line is the thing that luks creates when decrypting that device)
see, mapper one points to the same device
image

and then generated config is like this:

So you need to have the proper unencrypted device’s uuid in the mount section

Thanks @sirphobos !

That was exactly the problem. boot.initrd.luks.device."crypted".device needed to point to /dev/nvme0np2 but fileSystems."/home".device needed to point to /dev/dm-0.

I have to admit, I’m having a hard time understanding how folks solved this problem.

I happen to have a very similar situation to this problem. The problem occurred when I had to force restart on a system that was in the middle of running sudo nixos-rebuild switch --upgrade. Here’s what my situation looks like:

<<< NixOS Stage 1 >>>

loading module btrfs...
loading module dm_mod...
running udev...
Starting systemd-udevd version 254.6
Passphrase for /dev/disk/by--uuid/b90c2010-a53c-4f76-9802-95fdde248601 to appear. . . . -successs
Verifying passphrase for /dev/disk/by--uuid/b90c2010-a53c-4f76-9802-95fdde248601... - successs
starting device mapper and LVM...
Scanning for Btrfs filesystems
registered: /dev/mapper/luks-/b90c2010-a53c-4f76-9802-95fdde248601
mounting /dev/disk/by--uuid/b90c2010-a53c-4f76-9802-95fdde248601 on /...
[             8.956086] BTRFS error(device dm-0_: openk_ctree failed
mount: mounting /dev/disk/by--uuid/b90c2010-a53c-4f76-9802-95fdde248601 on /mnt-root/ failed: Input/output error

An error occurred in stage 1 of the boot process, which must mount the
root filesystem on `/mnt-root' and then start stage 2.  Press one 
of the following keys:
  r) to reboot immediately
  *) to ignore the error and continue

When I opt for “ignore the error and continue,” I’m notified the following:

Continuing...
BusyBox v.136.1 () multi-call binary.

Usage: switch_root [-c CONSOLE_DEV] NEW_ROOT NEW_INIT [ARGS]
Free initramfs and switch to another root fs:
chroot to NEW_ROOT, delete all in /, move NEW_ROOT TO /.
execute NEW-INIT.  PID must be 1. NEW_ROOT must be a mountpoint.

Would anyone be so kind to break the solution down for me?

That error is not indicative of any one problem. It’s the generic “oops something died” message for any failure during stage 1. So there isn’t one solution to break down for you. You’ll have to provide more of the log of what went wrong. But given that you mentioned a forced restart, I’m willing to guess your file system got damaged.

Yeah, it seems that way in the midst of rebuilding the NixOS.

I’ve updated my previous post with the specific error:

Good point. I didn’t think of that. I’ll start a new subject.