Nixos-anywhere bootstrap fails with network connectivity loss after kexec

Help
1

I’m having trouble with a nixos-anywhere installation on a new mini-pc. The bootstrap process fails right after the kexec reboot, and the machine becomes unreachable over the network, preventing the installation from continuing. I’d appreciate any help or suggestions to debug and resolve this.

The Problem

  • nixos-anywhere runs, copies keys, and successfully triggers the kexec reboot.
  • Immediately after the reboot into the NixOS installer, the machine drops off the network.
  • SSH attempts time out with “Operation timed out”, “Host is down”, or “No route to host”.

What I’ve Tried

  • Adding networking.useDHCP: I added networking.useDHCP = lib.mkDefault true; to the bootstrap configuration, hoping it would bring up the network interface (enp2s0) in the installer. This did not solve the issue.

My Questions for the Community:

  • Has anyone else experienced similar issues with nixos-anywhere and are there any known workarounds for network issues during the bootstrap phase? Any guidance would be greatly appreciated!

Appendix: Details and Logs

Hardware:

  • Machine: Minisforum NucBoxG5 (Intel N97, 12GB RAM)
  • Network: enp2s0 (wired), wlan0 (wireless)
  • Disk Config: Btrfs on LUKS

nixos-anywhere command:

nix run github:nix-community/nixos-anywhere -- \
  --generate-hardware-config nixos-generate-config ./hardware-configuration.nix \
  --flake .#generic \
  --target-host user@192.168.1.17 \
  --build-on remote

Log Output (showing the hang after kexec):
The logs show a successful handoff to kexec, and then connection timeouts.

...
machine will boot into nixos in 6s...
...
Connection to 192.168.1.17 closed.
Warning: Permanently added '192.168.1.17' (ED25519) to the list of known hosts.
...
ssh: connect to host 192.168.1.17 port 22: Operation timed out
ssh: connect to host 192.168.1.17 port 22: No route to host
ssh: connect to host 192.168.1.17 port 22: Host is down
...
2

How big the system closure? Disko has bad habit of pulling and building everything on the tmpfs since it creates the volumes last. I wonder if the issue is the tmpfs filling up and the system crashing because of it.

3

ip addresses can change after kexec when using DHCP, which is likely the issue you see here. If you can figure the new ip address, you can continue with nixos-anywhere against the new address.

4

unlikley 12gb is plenty. Also it’s not failing at the disko step.

5

My recommandation since you have physical hardware is to avoid kexec and use this image: GitHub - nix-community/nixos-images: Automatically build (netboot) images for NixOS [maintainer=@Mic92]
This also allows you to retry.

6

I’m not sure about this. Because when I reboot the target machine after the failure, I can ssh into it again using the same IP.

Yes, I’ll probably go down this route eventually. I had a great experience using nixos-anywhere when I set up my Hetzner VPS a few months ago, so I thought the same thing could work again, but I guess my networking is not reliable enough for this. Also I was hoping to avoid buying a USB drive lol

7

We did a couple of air-gapped installs with nixos-anywhere (with kexec) and always used IPv6 link-local. This way there is no need to deal with DHCP or other fancy stuff like DNS and such…

8

I had this problem recently. I’ve used nixos-anywhere successfully on vps machines. I wanted to experiment with different configurations on a spare laptop and experienced the same connection errors after kexec. My dhcp server (opnsense) was assigning a new IP address. I resolved the connection error by assigning a static IP based on the hardware address.

9

Thanks for this suggestion. I just tried it and unfortunately it didn’t change anything. At least this confirms that DHCP is not the culprit. For some reason once the target host goes into kexec it completely loses network connectivity. I guess I’ll have to go down the ISO route.

10

I think Nvidia graphics cards also don’t help with having to do kexec.