Hi,
This is a long post but hopefully it helps someone else.
I think I got a bit further. After creating a zfs pool the drive id’s disappear. Compare the values of blkid before and after:
blkid after fdisk:
blkid | grep sdb
/dev/sdb1: PARTUUID="a82a01eb-a38b-7842-9649-dd2762725ca7"
/dev/sdb2: PARTUUID="7b4a6cb8-2853-fa41-9d20-f4ac9acaeb5e"
/dev/sdb3: PARTUUID="50b7d816-5415-fd4c-acb2-7b96af52b648"
After zpool create:
DISK=/dev/sdb3
zpool create -O mountpoint=none -O atime=off -O compression=lz4 -O xattr=sa -O acltype=posixacl -o ashift=12 -R /mnt zpool $DISK
blkid | grep sdb
# nothing returned
zpool status
pool: zpool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
zpool ONLINE 0 0 0
sdb3 ONLINE 0 0 0
errors: No known data errors
After creating a dataset:
zfs create -o mountpoint=legacy zpool/tank
blkid | grep sdb
/dev/sdb3: LABEL="zpool" UUID="13714934573686082273" UUID_SUB="6456259992175977062" BLOCK_SIZE="4096" TYPE="zfs_member"
Now it has a LABEL and UUID. Checking with /dev/disk:
ls /dev/disk/by-uuid/
2178-694E 44444444-4444-4444-8888-888888888888 b6bca834-ccda-492a-88fe-48a233295f09
There is no matching UUID. And by label:
ls /dev/disk/by-label
FIRMWARE NIXOS_SD
There is no matching label. By ID:
ls /dev/disk/by-id/ | grep HGST
ata-HGST_HDN726040ALE614_K7GX5ZBL
ata-HGST_HDN726040ALE614_K7GX5ZBL-part1
ata-HGST_HDN726040ALE614_K7GX5ZBL-part2
ata-HGST_HDN726040ALE614_K7GX5ZBL-part3
ata-HGST_HDN726040ALE614_K7GX5ZBL-part3 should be the right partition.
Now if I mount the dataset:
mount -t zfs zpool/tank /mnt/tank
blkid | grep sdb
/dev/sdb3: LABEL="zpool" UUID="13714934573686082273" UUID_SUB="6456259992175977062" BLOCK_SIZE="4096" TYPE="zfs_member"
The /dev/disk devices haven’t changed after the mount.
So I’ll go back and create the pool by partuuid instead:
umount /mnt/tank
zpool destroy zpool
zpool create -O mountpoint=none -O atime=off -O compression=lz4 -O xattr=sa -O acltype=posixacl -o ashift=12 -R /mnt zpool $DISK
zpool status
pool: zpool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
zpool ONLINE 0 0 0
ata-HGST_HDN726040ALE614_K7GX5ZBL-part3 ONLINE 0 0 0
errors: No known data errors
blkid | grep sdb
/dev/sdb3: LABEL="zpool" UUID="13374479555073036372" UUID_SUB="17172025984875349708" BLOCK_SIZE="4096" TYPE="zfs_member"
Now with using the PARTUUID the device is showing after zpool create (where it wasn’t before) but the PARTUUID isn’t showing up in blkid or /dev/disk/by-partuuid.
ls /dev/disk/by-partuuid/
0d1e08a3-01 2178694e-01 2178694e-02
So it seems impossible for the mounter to find it. Let’s continue anyways:
zfs create -o mountpoint=legacy zpool/tank
blkid | grep sdb
/dev/sdb3: LABEL="zpool" UUID="13374479555073036372" UUID_SUB="17172025984875349708" BLOCK_SIZE="4096" TYPE="zfs_member"
But /dev/disk/by-uuid doesn’t have the above. Now the label appears.
ls /dev/disk/by-label
FIRMWARE NIXOS_SD zpool
The partuuid is missing:
ls /dev/disk/by-partuuid/
0d1e08a3-01 2178694e-01 2178694e-02
ls /dev/disk/by-uuid
13714934573686082273 2178-694E 44444444-4444-4444-8888-888888888888 b6bca834-ccda-492a-88fe-48a233295f09
But id is there:
ls /dev/disk/by-id | grep HGST
ata-HGST_HDN726040ALE614_K7GX5ZBL
ata-HGST_HDN726040ALE614_K7GX5ZBL-part1
ata-HGST_HDN726040ALE614_K7GX5ZBL-part2
ata-HGST_HDN726040ALE614_K7GX5ZBL-part3
So lets go back and create the pool with id:
umount /mnt/tank
zpool destroy zpool
DISK=/dev/disk/by-id/ata-HGST_HDN726040ALE614_K7GX5ZBL-part3
zpool status
pool: zpool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
zpool ONLINE 0 0 0
ata-HGST_HDN726040ALE614_K7GX5ZBL-part3 ONLINE 0 0 0
errors: No known data errors
blkid | grep sdb
/dev/sdb3: LABEL="zpool" UUID="15388353107909713984" UUID_SUB="14747893016884677337" BLOCK_SIZE="4096" TYPE="zfs_member"
ls /dev/disk/by-id |grep part3
ata-HGST_HDN726040ALE614_K7GX5ZBL-part3
zfs create -o mountpoint=legacy zpool/tank
mount -t zfs zpool/tank /mnt/tank
Add to configuration.nix
boot.zfs.devNodes = "/dev/disk/by-id";
Then re-run nixos-generate-config and it adds:
fileSystems."/mnt/tank" =
{ device = "zpool/tank";
fsType = "zfs";
};
Then nixos-rebuild switch and reboot.
Then on reboot the mount fails. If I go into single user mode and do a blkid only the drive (/dev/sdb) shows up with /dev/sdb TYPE="zfs_member". In /dev/disk/by-id only the hard drive itself shows up (ata-HGST_HDN726040ALE614_K7GX5ZBL) and not the partitions (ata-HGST_HDN726040ALE614_K7GX5ZBL-part3). So it looks like it can’t be mounted.
I tried zpool import -d /dev/disk/by-id and then it imports ata-HGST_HDN726040ALE614_K7GX5ZBL (not the partition part-3) and then it is CORRUPTED.
So maybe ZFS over USB is just a "bad idea"™? Any ideas for what might lead to a solution?
Glen