I need this for impermanence, and I don’t want to depend on a filesystem with a sub-volumes feature; so I can’t mount a device directly, basically I need to mount a sub-directory as the sysroot. The above config is working fine with the legacy initrd system, but postDeviceCommands is now deprecated and I’d like to port this to a systemd/initrd service.
The thing is I’m having a hard time figuring out how/when I can run my little script so that the bind-mount specified under fileSystems can proceed. I guess a bind-mount for the sysroot is not quite supported anyway, and I’m OK to write some more involved config since my use-case is weird, but I need some help.
I use this for impermanence with zfs. You might be able to adept it. The hardest part will be to figure out the right dependencies so it is executed at the time you need it.
boot.initrd.systemd = {
enable = true;
services.rollback = {
description = "Rollback root filesystem to a pristine state on boot";
wantedBy = [
# "zfs.target"
"initrd.target"
];
after = [
"zfs-import-rpool.service"
];
before = [
"sysroot.mount"
];
path = with pkgs; [
zfs
];
unitConfig.DefaultDependencies = "no";
serviceConfig.Type = "oneshot";
script = ''
zfs rollback -r rpool/local/root@blank && echo " >> >> rollback complete << <<"
'';
};
};
The same device can actually be mounted multiple times for most file systems, and X-mount.subdir= works for all file systems, even those without subvolumes (it just causes mount to make the mount in a separate namespace and then bind mounts the subdir to the final mountpoint).
Thanks! I did try something very close, but I can’t manage to run my service at the right time, since the bind-mount I have under fileSystems."/" seems to run very early (aka. I guess specifying a bind-mount there breaks some dependencies in the new initrd/systemd).
Thanks! Unfortunately I also tried this (before deciding to use bind-mounts). But the feature is flagged as experimental in the mount manpage (and as you said the implementation seems quite involved). Do you know whether I could bypass fileSystems."/" entirely and provide a custom service for initrd/systemd in which I would mount the sysroot myself?
It’s been around for almost 5 years. I wouldn’t say the implementation is “quite involved”; it’s really the same exact thing as what you’re trying to do instead, just in the mount command and with a little extra element of namespacing to make it atomic. I would just use it if I were you.
Anyway if you must, I would probably just do this:
# `fileSystems` always go under `/sysroot`, so `mounts` (or initrd's `/etc/fstab`)
# is the best way to mount stuff outside `/sysroot`.
boot.initrd.systemd.mounts = [
{
what = "/dev/foo";
where = "/mnt-tmp";
}
];
fileSystems."/" = {
fsType = "none";
device = "/mnt-tmp/current";
options = [ "bind" ];
};
fileSystems."/nix" = {
fsType = "none";
device = "/mnt-tmp/nix";
options = [ "bind" ];
};
systemd will automatically make sure that /mnt-tmp is mounted before /sysroot or /sysroot/nix. And if you need to run a script in between, you can add:
boot.initrd.systemd.services.foo = {
requiredBy = [ "initrd.target" ];
after = [ "mnt\\x2dtmp.mount" ];
before = [ "sysroot.mount" ];
...
};
Thanks a lot for all the details! I took some time to test everything and do some quick debugging, including via the emergency shell inside initrd.
For some reason, the X-mount.subdir= mount option seems broken inside initrd (and I’m quite sure it’s the same with scripted initrd since that was the first option I tried before settling for the bind-mount hack).
Your solution using boot.initrd.systemd.mounts is also not working, but I have a hard time figuring why (the associated Nix code is quite hard to read, and I don’t know systemd well).
Specifically, I just tried to boot with this simple config (without a custom script nor anything fancy):
.. from what I can tell from the emergency shell, /volume is never created, instead I can see that /sysroot/volume/nix has been created as an empty directory. /sysroot/volume is empty besides the empty nix directory.
If I don’t sort this at least I know I can easily switch to tmpfs for the root FS, do a bunch of custom bind-mounts and call it a day, so not a big deal, but I’d love to learn more.
Can you elaborate on this? Why do you think it’s broken? Like what error message do you get using it? X-mount.subdir= definitely does work in systemd initrd, and changes were made to support it in scripted stage 1 back in october. I really think you should figure out what’s not working about X-mount.subdir= because it is very much the right answer to this problem.
Oh right, I forgot that fileSystems / systemd-fstab-generator prefixes /sysroot to the device for bind mounts. So it’s trying to do the equivalent of mount --bind /sysroot/volume/current /sysroot rather than mount --bind /volume/current /sysroot. Hm, I’m actually not sure how to do this without doing something more complicated then… other than X-mount.subdir=
Okay so X-mount.subdir is working just fine. However I need to run a custom script before mounting sysroot. The following doesn’t work:
boot.initrd.systemd.mounts = [
{
what = "/dev/foo";
where = "/volume";
}
];
boot.initrd.systemd.services.foo = {
requiredBy = [ "initrd.target" ];
after = [ "volume.mount" ];
before = [ "sysroot.mount" ];
...
};
My custom service does run, but /volume is not mounted (as if mounts was not defined at all). When I try to mount it manually inside my script I get Can't lookup blockdev.
I’m using NixOS with a quite exotic 
