Stuck in Grub Cli

Hello, I just changed my default shell from zsh to fish, and rebuild my nixos configuration. I don’t think that is the cause, but it was the last thing I did before this problem. After rebooting, I am instantly stuck in the grub-cli, and can’t even rollback anymore because of that.

I would like to know:

  • how do i fix this?
  • why was this caused?
  • would it be a good idea to use systemd-boot to maybe avoid this problem?

I never had this problem before and this is the first time I got problems with the bootloader. If you know how to fix this it would be a huge help because I really don’t want to reinstall nixos without knowing what caused this and how to fix it. Thanks for your help.

  • I use flakes with the unstable-branch if it matters

How did you do so?

Any errors you can share?

I used the following code snipped from the NixOS Wiki

programs.bash = {
  interactiveShellInit = ''
    if [[ $(${pkgs.procps}/bin/ps --no-header --pid=$PPID --format=comm) != "fish" && -z ''${BASH_EXECUTION_STRING} ]]
    then
      shopt -q login_shell && LOGIN_OPTION='--login' || LOGIN_OPTION=""
      exec ${pkgs.fish}/bin/fish $LOGIN_OPTION
    fi
  '';
};

After rebuilding without encountering errors I just rebooted, no errors or something like that just instantly the grub cli.

The NixOS Wiki is at fish - NixOS Wiki

I doubt a change in userland would affect the bootloader, are you sure you are being thrown into the GRUB rescue shell and not the system recovery mode?

1 Like

Like I said, I also don’t think it has anything todo with fish, I never encountered the GRUB rescue shell or the system recovery mode before, so I can’t say wich of the two is it, but since it says GRUB I would guess it is the GRUB rescue shell, here is a picture:

I tried a bit of stuff the past few hours, notably:

set root=(hd0,gpt1)  # ESP partition with /kernels
linux /kernels/<hash>-linux-xanmod-6.12.19-bzImage root=/dev/sda1 # I also tried nvme0n1p2 and sda2
initrd /kernels/<hash>-initrd-xanmod-6.12.19-initrd
boot

And after doing that, it seems like something is happening? It sadly says:

An error occured in stage 1 of the boot process, which must mount the root filesystem on /mnt-root and then start stage 2, Press one of the following keys:

r) reboot
*) continue and ignore

and it also says a bit above that stage 2 intit script /mnt-root//init not found

Please not that I just tried “something” because I wanted to see if I can get it running temporarily. I am pretty sure that I made some error.

I was very close to crashing out this time and giving up, but I think I’ve managed to get it working again. It’s still building, but here’s what I did:

  1. Used the live environment from a USB stick (e.g., installer ISO).
  2. Closed the installer and connected to the internet.
  3. Opened the terminal.
  4. Mounted the file system (use lsblk if you’re unsure which device to mount):
  • sudo mount /dev/nvme0n1p2 /mnt
  • sudo mount /dev/nvme0n1p1 /mnt/boot
  • sudo nixos-enter
  1. Used an old configuration. For example, the original config is still available at /etc/nixos/configuration.nix on my system.
  2. Ran the following command to reinstall the bootloader:
  • NIXOS_REINSTALL_BOOTLOADER=1 nixos-rebuild switch --install-bootloader
  1. You might also need to specify the configuration file with -I nixos-config=/etc/nixos/configuration.nix.

Additionally, the following service encountered a build failure, so I temporarily disabled it:

  • services.logrotate.enable = false

Now I am waiting and hoping for it to work, this is the best attempt until now.

1 Like

Yea, you’re missing the kernel params. NixOS relies on an init= kernel param that is specific to each generation.

If grub isn’t doing this correctly automatically, I’m afraid your grub.cfg was broken somehow. Can you boot the live ISO, mount your partitions, and look at /boot/grub/grub.cfg?

1 Like

I fixed it wait I will make one final message for the future so people can just read it, I think I probably did something crappy, so if you can explain further what you mean I will tag it as the solution.

How I fixed this

  1. Booted into the live environment from a USB stick (e.g., installer ISO).
  2. Closed the installer and connected to the internet.
  3. Opened the terminal.
  4. Mounted the file system (use lsblk if you’re unsure which device to mount):
  • sudo mount /dev/nvme0n1p2 /mnt
  • sudo mount /dev/nvme0n1p1 /mnt/boot
  • sudo nixos-enter
  1. Used an existing configuration. For example, my original config is still located at /etc/nixos/configuration.nix.
  2. Ran the following commands to reinstall the bootloader:
  • unset SUDO_USER (needed to bypass PAM)
  • nixos-rebuild boot --install-bootloader
  1. You may also need to specify the configuration file with -I nixos-config=/etc/nixos/configuration.nix.

Additionally, I encountered a build failure with the following service, so I temporarily disabled it:

  • services.logrotate.enable = false

Once the build is complete, you may see the following messages:

  • System has not been booted with systemd as init system (PID 1). Can’t operate.
  • Failed to connect to system scope bus via local transport: host is down.

This is normal. By the end, it should confirm that a new boot entry has been created. Reboot your system and, at the GRUB CLI, type exit to reach the bootloader selection screen. From there, select the newly created entry, and your system should boot correctly.

I have never needed this when using nixos-enter, so that’s weird. In fact you shouldn’t need sudo at all once you’re in the nixos-enter shell.

The environment variable you were looking for was NIXOS_INSTALL_BOOTLOADER=1, not REINSTALL. But it was made redundant by --install-bootloader, which sets that variable before calling switch-to-configuration (the part that actually uses it), so it was ok.

Glad you got it fixed :slight_smile:

2 Likes

Oops, I meant to say that NIXOS_INSTALL_BOOTLOADER=1 was a typo. I’ll correct that.

I needed to use SUDO_USER because, at the end of the rebuild, PAM would throw an error (I assume that’s where the sudo is executed).

Thanks for helping me!

Do you maybe have an idea what could’ve broke the grub.cfg? I think I will stick with systemd-boot for now anyways although I would be interested for the reason.

nixos-rebuild doesn’t use sudo, unless you passed --use-remote-sudo or well, called it using sudo. That shouldn’t be necessary though since you’re already root.

1 Like

That is weird, without it I always got an error from the PAM. When I have try I will try to rebuild again from chroot. Thanks for the information though.

Just in case that’s not just shorthand, I want to be clear that nixos-enter is more than just chroot, and a plain chroot will probably lead to unexpected problems.

3 Likes

Yes sorry I should articulate myself more accurate :nix_parrot: