Nix iso unable to boot in UEFI mode (but other distros can)

Help
1

I am currently trying to install nixos21.11 on real hardware. However, the installer crashes right after the graphical menu. The menu stays there forever, and if I press any button, I get thrown right back to the UEFI boot menu. No errors, not even a tiny bit of console output. I does not matter which option I choose, the behavior never changes. I have the following hardware:

  • Xeon E3 1245 v5

  • Asrock E3C236D2I (has IPMI with AST2400 graphics)

  • 16GB ECC DDR4

  • 2x14TB HDD + 2x1.8TB SSD + 512GB boot NVMe (needs UEFI)

I am using the IPMI web console to install, this works fine with other operating systems (fedora 34, 35 and also debian). I also tested the nixos iso (works perfectly fine inside a VM) and even the boot stick on another system.

After some more investigation - it can boot if I set the bios options to only allow legacy (BIOS mode).

However all other linux distros I have tried so far work fine in UEFI mode (I currently have fedora iot 35 installed). I can also use my nixos boot stick inside a UEFI virtual machine just fine. I am not sure where to even start debugging this. I do need UEFI for the NVMe drive.

2 Likes
2

Just to see if it has something to do with the newest iso 21.11, try 21.05?

For example: https://channels.nixos.org/nixos-21.05/latest-nixos-plasma5-x86_64-linux.iso

If that works, you can upgrade it…

1 Like
3

I tried the 21.05 iso I still had laying around (is there an official place to get the older releases?). It has the exact same behaviour as the 21.11 iso sadly.

4

Hmmm, so older did not work, maybe newer?

Via https://hydra.nixos.org/project/nixos

I found: Hydra - Build 160970723 of job nixos:unstable-small:nixos.iso_minimal.x86_64-linux

Or by looking around, maybe there is an other one you like better

5

Thanks for the link!

I tried this build: Hydra - Build 161083266 of job nixos:unstable-small:nixos.iso_minimal.x86_64-linux

But sadly, no luck either. Same problem. I wonder what could cause something like this to happen.

6

it certainly a commit! You’ll have to bisect the build , and generate installation iso’s until you find the culprit. It’s not going to be quick because you’ll have to boot on bare metal.

This is a chance the nixos has never worked on your motherboard bios and the other distro’s have either patched or worked around a quirky or broken bios.

We really need a bare metal build farm, to test all things like this, there’s only so much you can do with qemu virtual machines. It’s been done in the past, especially in the embedded space.

Without this hardware i can’t debug, as all my machines have been fine with UEFI so far.

linuxPackages.rtl88xxau-aircrack: fix build for linux 5.15 by MatthewCroughan · Pull Request #145697 · NixOS/nixpkgs · GitHub , highlights a possible future for a bare metal farm for testing boot, drivers, kernels, performance, etc etc etc… per commit! This is actually achievable with the right infrastructure.

image
image1024Ă—1024 39.8 KB

7

Does your IPMI give you serial? If so, try pressing e on the menu, add console=ttyS0 to kernel command line, and see if you get any messages.

2 Likes
8

I do have SoL on this motherboard.
Pressing “e” crashes/instantly closes the installer menu and brings me back to the uefi boot menu. Same if I try to press “t” for the text boot menu. The graphical menu never shows up in Serial mode, but otherwise everything behaves the same.

I think it might be the menu that is causing problems. It probably never gets as far as starting the actual boot sequence.

Anywas, I will probably will sell this board. Already had a lot of other problems with it (its very very picky with RAM and the IPMI software needs java versions that can only be found on archive sites at this point).

9

Nixinator…shudders…

10

Since it sounds like the problem is in the GRUB boot menu, if you want to give it another try before you sell the board, you could try creating a custom live image that used systemd-boot instead of GRUB. (boot.loader.systemd-boot.enable = true;) But most other distros use GRUB as well, so even if that worked it wouldn’t explain why other distros worked for you but NixOS didn’t. :frowning:

2 Likes
11

I just tried that (amazing how easy it is to build a custom ISO), same problem. However the boot screen looks absolutely identical. Is that one line enought to switch to systemd-boot?

2 Likes
12

boot.loader.systemd-boot.enable = true; for ISO won’t do anything. You can try doing this instead: NixOS 23.11 manual | Nix & NixOS
Use any working Linux LiveCD ISO and after installing nixos-install-tools follow the usual installation process.
systemd-boot is used by default for UEFI installation, so if the issue is really with GRUB, it should work out-of-the-box

2 Likes
13

excuse me if it’s a dumb question but, do you have enable safe boot? (It’s also called secure boot in some bios)

14

Secure boot should refuse to even bring up the graphical menu, since that is part of the un-signed binary.

1 Like
15

I assume that my board does not even support secure boot. It has no options whatsoever inside the UEFI that would allow me to enable/disable secure boot. I searched the manual very thoroughly because I also assumed the issues could be caused by secure boot, however I could not find anything.

16

Hi,

I have the exact same problem with a Zimaboard and NixOS 22.11 USB key. The key works well on other computers, just on that one any option I select in the “NixOS boot selection” after the BIOS boot device selection prompt, immediately brings me back to the BIOS boot device selection prompt. I wasn’t able to figure out a way to extract an error message. I’ve tried changing a few options in the BIOS, but no luck. I’ve tried NixOS unstable, same.

A USB key with Debian or Arch Linux works just fine on the same miniPC. Did you manage to work around the problem in that thread?

Thanks,

17

Replying to myself.

Here is what I did to workaround the issue:

  1. Boot into Debian using their USB installer
  2. Install Nix in Debian sh <(curl -L https://nixos.org/nix/install) --daemon
  3. From there, build a NixOS to kexec (see documentation NixOS 23.11 manual | Nix & NixOS): nix-build -A kexec.x86_64-linux '<nixpkgs/nixos/release.nix>'
  4. apt install kexec-tools
  5. cd result && ./kexec-boot
2 Likes
18

Have just been through this issue with identical behaviour as described in OP.

What solved the issue for me (believe it or not) is booting the media from the BIOS boot selection menu.

So, if the NixOS install media is the default boot device - freezes at menu as described.
If the NixOS install media is manually selected after invoking the BIOS boot menu - works as intended.

Motherboard in question is a Chinese “X99” board with 6 core Xeon.

1 Like
19

Can we open an issue in nixpkgs to collect the exact hardware that does not work with our UEFI ISO?

1 Like
20

I realize this is a very old thread but I report the same behavior with 23.05 on a device I have (a GPD Win 3 handheld gaming computer). I second the call to develop some sort of automated testing for this.

I’m no longer in a position to test it but at one point I had a Windows installation on the internal drive, and the behavior I saw at that time was consistent with @ahen’s report about manual selection vs. default boot device. I actually managed to get into the installer once when I had it in that state, but, alas, did not succeed in completing the install.

I’m going to play around some more and report back.

1 Like