NixOS unbootable after nix-collect-garbage -d: no boot menu, cannot chroot from live system

Help
1

After updating my NixOS configuration and running:

sudo nixos-rebuild switch
sudo nix-collect-garbage -d

my system became completely unbootable. I no longer even see a NixOS/systemd-boot menu. I’m now trying to recover from a Debian Live USB I happened to have around.

I suspect I deleted all system generations and the /nix/store contents that contained the boot configuration. I’m not sure of the cleanest recovery path.

System information:
NixOS was installed in UEFI mode, using systemd-boot. The output of lsblk -f is:

NAME   FSTYPE   FSVER            LABEL                  UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
loop0  squashfs 4.0                                                                                0   100% /run/live/rootfs/filesystem.squashfs
sda                                                                                                         
├─sda1 vfat     FAT32                                   C8DF-1D46                             473.4M     7% /mnt/boot
└─sda2 ext4     1.0                                     509efae4-ec18-4057-93d4-a0bf08037f85   57.4G    89% /mnt
sdb    iso9660  Joliet Extension d-live 13.1.0 kd amd64 2025-09-06-09-42-55-00                              
├─sdb1 iso9660  Joliet Extension d-live 13.1.0 kd amd64 2025-09-06-09-42-55-00                     0   100% /run/live/medium
└─sdb2 vfat     FAT12

I tried to mount the root and boot paritions then chroot into NixOS, but that didn’t work, likely because all system generations and the store were garbage-collected somehow. Is there another way I can reinstall NixOS in place? I’d love guidance on the cleanest way to rebuild a working NixOS system from this state.

1 Like
2

Chrooting into a nixos install doesn’t really work without some special tweaks. There’s a tool for it called nixos-enter which is available from nixpkgs, assuming you have a live environment with nix on it.

(Note: if you do a nixos-rebuild from inside nixos-enter, make sure it’s a boot, not a switch or test. Activating the new generation from inside nixos-enter fails rather badly since systemd isn’t running.)

3 Likes
3

Thanks so much for responding. I got a NixOS live USB and after mounting root and boot tried nixos-enter then nixos-rebuild boot, but I fear I may have messed my system up even further. I kept getting errors like

error: 1 dependencies of derivation '/nix/store/8xz4h050j0zpfg9k8w3n5z58siqincl8-unit-logrotate-checkconf.service.drv' failed to build copying path '/nix/store/zx39iyplbwxw8m679pvf2finix26wc45-mdadm-4.2' from 'https://cache.nixos.org'... error: 1 dependencies of derivation '/nix/store/7cqr1dalkwd7qw5rw3ydki3bg90ncrhs-system-units.drv' failed to build building '/nix/store/3lrlhinpz21yvbqkg8kh0d41kvd8p5h0-systemd-user.pam.drv'... building '/nix/store/94rn16l51phrj3slqrhllnhh5xqd87nh-systemd.drv'... building '/nix/store/lbgm59ik3wlwl72j48aqyxx7mfzhrycl-unit-dbus.socket.drv'... building '/nix/store/qjv6cw3cxv4b9rsdv529ig3byz07igb4-unit-nixos-fake-graphical-session.target.drv'... error: 1 dependencies of derivation '/nix/store/qx9jg4has5cg18knmws442gj5qsnz197-etc.drv' failed to build copying path '/nix/store/i0xyyzcm6x82n9r7i0yy92dv6sn8k02d-time-1.9' from 'https://cache.nixos.org'... copying path '/nix/store/dkxr3gjzs9hfjly054wmkddljs48ri6w-udev-rules' from 'https://cache.nixos.org'... error: 1 dependencies of derivation '/nix/store/r13rdxylqhqfp5gixgklxg0l0g2mqm5m-nixos-system-nixos-23.11.7870.205fd42

so I stripped my configuration.nix file down to this

# /mnt/etc/nixos/configuration.nix
{ config, pkgs, ... }:

{
  imports = [ ./hardware-configuration.nix ];
  services.logrotate.enable = false;
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  networking.hostName = "nixos";

  environment.systemPackages = with pkgs; [ vim git ];

  services.sshd.enable = true;
}

After I reran nixos-enter and nixos-rebuild boot it seemingly succeeded in rebuilding the bootloader but I still couldn’t see anything in my boot menu. All I want now is any NixOS system I can boot into, as long as I don’t have to wipe my hard drive and reinstall. Any advice or help would be appreciated.

4

What you’ve quoted doesn’t really mention the actual error that caused the problem. It’s just all describing the dependency chain of the error propagating, and other things that were happening in parallel. Regardless, it looks like the slimmed-down config got around this issue.

  • Did you mount everything correctly before doing the nixos-enter? Including /boot (as seen from inside the chroot)? If /boot wasn’t mounted you could get this behavior.
  • Try passing --install-bootloader to the nixos-rebuild boot as well. If the bootloader itself isn’t correctly set up, just recreating the menus won’t help.
1 Like
5

Again, thanks for the suggestions, and I apologize for my being a bit slow. I tried a couple of times and I’ve attached all the commands I ran and their output in my most recent attempt here plus my original configuration.nix. My goal is to get a bootable NixOS system that is close to that config, but I’m nervous to try any further because I don’t really know what I’m doing. If you were in my situation (i.e. in a fresh NixOS Live USB with a configuration.nix file like this in your Downloads folder from your git repo), what sequence of commands would you attempt in order to recover?

6

If I were in your case, I’d use a Live USB to backup all of my data to an external harddrive, and start the whole installation process from a fresh start.

7

I unfortunately don’t have a large enough external hard drive nearby to do that, and can’t really afford to take this to a shop. I’m a very broke student.

8

The commands look alright, but you should try --install-bootloader on your nixos-rebuild boot command, as I suggested above. Also, you took system.stateVersion out of your config when you tried to slim it down. Bad idea. Never modify or remove that setting, for any reason.

2 Likes
9

Could you explain how I am to check whether /boot is mounted in the correct location as you mention? I just mounted it to /mnt/boot which is what the wiki suggests.

After many other dead ends I went back to my original configuration and tried to install it as usual (it is part of a Git-controlled flake). The last session I tried is here; it seemed to succeed with some warnings but upon rebooting I still get no menu.

I pasted my hardware-config just in case that’s also useful, and would happily share any other information that might be pertinent.

10

The way you mounted /boot looks fine. The bootloader install seemed to work, too. Honestly I’m out of ideas on why it isn’t working. Maybe someone else knows something.

Just in case the obvious has been missed, let me double-check: You did remove the nixos install media before rebooting, right?

1 Like
11

Yes, I’ve been removing the live USB before trying to boot normally. The output suggests Nix is now taking the state version as 25.11 which I don’t understand.

12

That’s not the stateVersion, that’s the release number corresponding to the nixpkgs commit you built from. (And just to drive the point home in case you’re not aware: Don’t change stateVersion! Even when you upgrade to a new release. Just leave it be.)

2 Likes
13

If you mean the warning message, nixos will set stateversion to the current nixos release if you don’t set stateversion. Stateversion bumping itself will cause problems with some modules that use stateful data. Hence you always set it to a valid value and never touch it.

14

Well, it’s maybe worth taking a step back and looking at what you’ve got. What is in /boot? Are there viable grub/systemd-boot configuration files? What generations do they point at? Do those generations exist in /nix/store?

Also: It seems very unlikely to me that collecting garbage could cause this, more likely to be a boot loader problem. Especially given nixos-enter doesn’t appear to complain about issues with your store by your description.

So, what do you actually see on boot? Did you accidentally switch between UEFI and legacy boot in your BIOS? Or maybe it switched to a boot order that makes no sense?

1 Like
15

Thanks for taking a look. Here’s the output of another attempted recovery, where I tried to include the information you request. As far as I can tell nothing is amiss.

As for what I actually see on boot: I run a Lenovo Thinkpad X260. If I do not plug in a USB with a live system and press Enter to get into the boot device configuration, I stay stuck at the red Lenovo logo at the beginning of the boot sequence. The menu where I might pick different generations to launch doesn’t appear.

16

Yeah, that looks fine. Can you share your configuration? I’m blind

This honestly sounds like the firmware is just failing to start the bootloader. Any secure boot settings that were flipped on accidentally? Can you try a firmware update? What happens if you go into BIOS and use the boot override to select the on-disk system?

Wiping /boot completely and just reinstalling the bootloader may also be worthwhile. It’s obvious that stuff goes wrong well before stage 1, so I wouldn’t worry about reinstalling anything, it’ll just result in the same system or wiping /boot will have the same effect.