Every other boot on my server the EFI folder is missing under /boot it only contains the /boot/loader/random-seed. The system boots fine and works as expected. I only notice when I try to update the system because it fails with Could not find any previously installed systemd-boot which is understandable because it is missing. If I reboot the system the EFI folder apears again and I can update the system as expected.
Any hints on how to debug this further?
When the error happens, is your ESP actually mounted at /boot?
It seems like it.
mount | grep boot
/dev/nvme1n1p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
I unmounted it and remounted it again but it does not change anything.
The strange thing is that a reboot (or two) fixes the issue.
And journalctl shows that it actually does bot from there.
Mar 28 16:08:26 nixserve kernel: Command line: initrd=\EFI\nixos\gjbr2vdqgxi78w1pjhydpnfhwd60257r-initrd-linux-6.18.19-initrd.efi init=/nix/store/42f190f60a6fxgyj3sxlismbra6rlcfa-nixos-system-nixserve-26.05pre968305.fdc7b8f7b30f/init nohibernate root=fstab loglevel=4 lsm=landlock,yama,bpf
``Do you maybe have multiple devices competing for the device name you have configured for /boot?
e.g.
fileSystems."/boot" = {
device = "/dev/sda1";
...
};
would have inconsistent results across boots since sda can reasonably change to point to a different disk. Or even /dev/disk/by-uuid/asdf-asdf can have a race condition when two different devices have file systems with the same UUID, and similarly for /dev/disk/by-label/foo.
Thank you so much @ElvishJerricco. That was it. I setup mirrored drives and wanted to also rsync the boot partitions but never did. So I have two similar drives with the same partitions and it seems pure luck when the right one was mounted.