Since A) I’m kinda masochistic and B) I’m tired of Grub and C) A lot of new distros are moving to sd-boot anyway (playing catch-up with NixOS):
I set up a KVM install with 5-6 different distros and attempted to share both /boot and /home partitions
All the distros I’ve tried this silly trick with are OK with using a /dev/disk-by-uuid style of /boot or /boot/efi mount point.
EXCEPT for NixOS. I have manually mounted the “shared” /boot environment and umounted the partition that originally launched it
Now nixos-generate-config is actually creating two /boot mounts in hardware-configuration.nix. The second one is the “desired” volume on a different VM (e.g., /dev/sdg1), and nixos-rebuild {switch|boot} complete successfully and actually write to the items in /boot/loader and /boot/loader/entries on /dev/sdg1.
But after reboot, using that systemd-boot as launcher, the “old” /boot partition is mounted. This is the one on /dev/vdh where NixOS was initially installed
Anyone help me with this headscratcher? I would like to reclaim the 1G ESP but first I have to be certain it will boot OK without it. Currently have my doubts about that.
I also wonder if there would be any way to do a fresh “chroot” install with the “foreign” ESP? This may be the easy answer, as I’ve spent several hours on this rabbit hole.
If you have an existing installation of NixOS on an EFI system that currently boots using grub, and want to migrate to systemd-boot, you throw away the grub config, replace it with systemd-boot config, run nixos-rebuild boot --install-bootloader, and reboot.
Depending on the firmware you might have to go though the firmwares boot selection screen once to select systemd-boot rather than the old grub.
Once you confirmed that systemd-boot actually works, you can remove the grub remains using rm.
Thanks for taking the time to read and reply! Here is my lab experiment:
See if I can get multiple OSes (including NixOS) to use a shared /boot environment with sd-boot
Several of these OSes are still in the “Grub era” and have to be coaxed to use sd-boot, such as Fedora 44, Linux Mint 22.3, and Debian 13. Others, such as OpenSUSE Tumbleweed (and NixOS) have already made the transition, and use sd-boot by default in their installer
NixOS is the focus of this post, of course. I’ve managed to address the problem by manually mounting the “shared” /boot partition /dev/vdg1 and manually changing the UUID in hardware-configuration.nix, then running nixos-rebuild-switch. After a reboot, all is well and all updates and runs of nixos-rebuild boot will touch and update the shared boot environment.
However, it is troublesome to me that nixos-generate-config still picks up both /dev/sdg1 (shared) and /dev/sdh1 (the “local” boot partition declared at install time) and adds them both to hardware-configuration.nix.
I’m also interested in learning whether I can perform a fresh install of NixOS pointing to a shared sd-boot partition off the installation target? This would allow me to use the whole vdx for NixOS/swap, instead of dedicating 1GB for /boot.
Please share exactly how your drives are mounted at runtime of nixos-generate-config and what the generated HW config is.
I do not see a reason why it should generate 2 entries for the same mountpoint, unless you have indeed managed to mount 2 devices at the same location shadowing each other.
In general I have not yet seen problems when having a common ESP for several different Linuxes. It might become more complicated, if you want to have them all in the same systemd-boot menu.
Mostly because of the annoying fact that you have to scroll through all the generations first before being able to select some other Linux, but also because of having to manually configure the boot entries for systemd-boot.
There’s no such thing as a “foreign” ESP partition. Nowadays people usually have one and share it between all OSes, but you can just as easily have an ESP partition per OS, or mix and match. Alas, regardless of what you’re doing to make nixos-generate-config generate two entries for /boot, it doesn’t really matter - it’s meant to be a starting-point, not ground truth for the config. It often misses stuff or generates config that’s not-exactly-correct and there’s nothing wrong with editing it to meet your needs.
Yes things are much calmer in the lab world. I did use a VM that initially had GRUB as the boot loader, so the odds are good that when I manually pointed /boot at the shared gummiboot loader, nixos-generate-config got confused and wrote two different /boot entries in hardware-configuration.nix.
Food for thought: but it looks too complex for a homelab, so I decided to just boot a second NixOS instance off a discrete /boot partition if I needed to run both.
Still a little unclear after running several non-NixOS distros, the absolute difference between mounting the vfat partition on /boot versus /boot/efi. I guess the latter is considered grub-legacy, but some distros use it for sd-boot anyway.