NixOS hangs on boot

I upgraded nixos on 13/03, since then i can’t use my laptop because it freeze on boot.

i try using an live usb with nixos-minimal-22.11.3092.970402e6147-x86_64-linux.iso to read the logs, but i had the same problem when booting, i had to use an older release (nixos-minimal-22.11.1777.cdead16a444-x86_64-linux.iso) to enter.

Anyone knows how to solve this?

Have you tried choosing a previous generation from the boot menu, such as the one just before your upgrade?

If that works, try to upgrade again with sudo nix-channel upgrade and sudo nixos-rebuild boot. NixOS builds around the 13th seem to be problematic for some of us. (Something bad around kernel v5.15.99.)

1 Like

Sadly there is only one generation.

I tried this using the live usb, with nixos-enter and sudo nixos-rebuild switch --options sandbox false but it gives me the next error:

I tried sudo mount /dev/sda1 /boot/efi in an attemp to fix it, but it didnt work:

I tried using --upgrade but it gives me this error:

I also tried with boot instead of switch, at first seems to work, but after rebooting the generation doesn’t appear

Do you know if it’s possible to downgrade the kernel from a live usb?

I’m struggling to imagine how you could “upgrade” and not have the prior generation. But I suppose it doesn’t matter – if it’s gone, then it’s gone.

I’m not sure quite what you mean by “downgrade the kernel from a live usb”. Running from a live usb should give you enough control to change pretty-much anything on the destination system. What you want to change will affect how to do it. Working from a live usb is outside my experience, but I can try to offer some guidance if you have the patience.

To figure this out, we may have to dig into details of how the system boot process works. You might prefer to just reinstall rather than wading through these details – especially if you have backups of any data you care about. (If you don’t have backups, maybe you can use the live usb to copy what matters most.)

NixOS symlinks to prior generations in /nix/var/nix/profiles/. When configuring the boot loader, corresponding files are written to /boot/loader/entries/. Note that these are directories in separate partitions on the boot drive: fat32 for /boot and some linux type such as ext4 for /. (I am assuming your system is configured to use systemd-boot as the boot loader; if not, this info may not apply.)

When booting the live usb, I would expect these directories represent the usb drive and that your target drive probably is not even mounted. The nixos-enter wiki page has good info as far as it goes, but links to the NixOS manual at a level that doesn’t help so much. I think you have to manually configure /mnt with your installation target filesystems before running nixos-enter. This means mounting relevant partitions.

You should be able to get info about your laptop drive by running lsblk. That should display a list of block devices something like this

sda           8:0    1    0B  0 disk
├─sda1        8:1    0  512M  0 part /boot
└─sda2        8:2    0  16G   0 part /
nvme0n1     259:0    0  1.8T  0 disk
├─nvme0n1p1 259:1    0  512M  0 part
└─nvme0n1p2 259:2    0  1.6T  0 part

Your version will be different. What’s important is that there is one device showing a mountpoint of / – device sda in my list. This is the current root filesystem, which should be your live usb.

The other device – nvme0n1 in my list – is your laptop drive. It should show at least two partitions – one with a size less than 1G and one much larger. Typically the smaller one is first and is used for /boot and the larger one is / for the new OS.

To use nixos-enter, you first mount your target root partition at /mnt and then mount your target /boot partition at /mnt/boot. For my nvme0n1 device the commands are

mount /dev/nvme0n1p2 /mnt
mount /dev/nvme0n1p1 /mnt/boot

After mounting those partitions, you can use ls to explore what files are actually there. Hopefully you will find /mnt/nix/var/nix/profiles/ contains more than one system generation. I expect that you will find that /mnt/boot/loader/entries/ does not match the profiles.

With those partitions mounted, running nixos-enter should get you a shell where you can use nix commands to fix the system. (You may have to do extra work to get an Internet connection.) If nixos-rebuild does not work, it’ll help to know more about what files you see.

1 Like

i have configured nix.gc to delete all prior generation after upgrading.

$ ls /mnt/nix/var/nix/profiles/
per-user  system  system-134-link
$ ls /mnt/boot/loader/entries/
nixos-generation-108.conf.tmp  nixos-generation-109.conf.tmp  nixos-generation-110.conf.tmp  nixos-generation-134.conf.tmp

Should i try bootctl install?

Thanks for clearing up that little bit of mental anguish. :wink:

Yes, do try bootctl install as suggested in that GitHub thread. That is a good find.

If that alone does not work, add boot.loader.efi.efiSysMountPoint = "/boot/efi"; to your config (also from that thread). I would hope this isn’t necessary – I do not have it in my config – but if it is, I’ll dig into the source to learn why.

1 Like

Sadly that didn’t work, but sudo mount /dev/sda2 /mnt && sudo mount /dev/sda1 /mnt/boot/efi && sudo nixos-install did the trick.

Thanks for the help :+1: :+1:

Hosted by Flying Circus.