NisOS 25.11 fails to boot after creating new generation

I’m running NixOS 25.11 (channel nixos-25.11), and have the following boot snippet in my configuration.nix:

  boot = {
    loader.systemd-boot.enable = true;
    loader.efi.canTouchEfiVariables = true;
    supportedFilesystems = [ "ntfs" ];
  };

Yesterday I run sudo nixos-rebuild boot -–upgrade. Trying to boot the generated system config, the boot loader gives the error “…-linux-6.12.75-bzImage.efi: unsupported”. The /boot partition has plenty of free space. The previous generation from ca. 10 days ago boots as expected, and I’ve done no changes to the configuration.nix file.

Has anything related to the boot loader changed over the last few days? Any suggestion?

No.

Do you use git? Can you look at the exact diff? It sounds like your kernel was compiled with initrd disabled, maybe you did make a change to the kernels you’re booting.

Share your full config if you want more than educated guesswork.

Thanks for your reply. The configuration git diff is empty. I only upgraded the channel. I have no config at all regarding the kernel. Due to the upgrade, the kernel changed from 6.12.74 to 6.12.75.

You might just be unaware, share your config please. Worst case we can build it and run it locally to reproduce your issue.

1 Like

OK. Here my configuration.nix:

{ config, pkgs, ... }:

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
      ./qemu-kvm.nix
    ];

  ###########################################################333
  # Basic setup
  ###########################################################333

  boot = {
    loader.systemd-boot.enable = true;
    loader.efi.canTouchEfiVariables = true;
    supportedFilesystems = [ "ntfs" ];
  };

  nix = {
    extraOptions = ''
      experimental-features = nix-command flakes
    '';
   };

  fileSystems."/home" = {
    device = "/dev/pool/home";
    fsType = "ext4";
  };

  networking.hostName = "orione"; # Define your hostname.
  # networking.wireless.enable = true;  # Enables wireless support via wpa_supplicant.

  # Set your time zone.
  time.timeZone = "CET";

  # The global useDHCP flag is deprecated, therefore explicitly set to false here.
  # Per-interface useDHCP will be mandatory in the future, so this generated config
  # replicates the default behaviour.
  networking.useDHCP = false;
  networking.interfaces.enp4s0.useDHCP = true;
  # Allow containers to use external network
  networking.nat.enable = true;
  networking.nat.internalInterfaces = ["ve-+"];
  networking.nat.externalInterface = "enp4s0";
  networking.networkmanager.unmanaged = [ "interface-name:ve-*" ];

  # Select internationalisation properties.
  i18n = {
    defaultLocale = "en_GB.UTF-8";
    # required since nixos-25.11
    inputMethod = {
      enable = true;
      type = "ibus";
    };
  };
  console = {
    keyMap = "sg-latin1";
  };

  # Enable the X11 windowing system.
  services.xserver.enable = true;

  # Enable the GNOME Desktop Environment.
  services.desktopManager.gnome.enable = true;
  services.displayManager.gdm.enable = true;
  
  # Configure keymap in X11
  services.xserver.xkb.layout = "ch";

  # Enable CUPS to print documents.
  services.printing.enable = true;

  # Enable touchpad support (enabled default in most desktopManager).
  services.libinput.enable = true;

  # Fonts
  fonts = {
    fontDir.enable = true; # add dir in /run/current-system/sw/share/X11/fonts
    enableDefaultPackages = true;
    enableGhostscriptFonts = true;
    packages = with pkgs; [ source-code-pro ];
  };

  ###########################################################333
  # Users
  ###########################################################333
  
  # Define a user account. Don't forget to set a password with ‘passwd’.
  users.users.user1 = {
    isNormalUser = true;
    home = "/home/user1";
    homeMode = "750";
    description = "User 1";
    uid = 1000;
    extraGroups = [ "wheel" "networkmanager" "users" "libvirtd" ]; # Enable ‘sudo’ for the user.
  };

  ###########################################################333
  # Global software
  ###########################################################333

  # Required for zoom
  nixpkgs.config.allowUnfree = true;
  
  # List packages installed in system profile. To search, run:
  # $ nix search wget
  environment.systemPackages = with pkgs; [
    (aspellWithDicts (d: [d.en d.it d.de d.fr]))
    audacity
    nodePackages_latest.bash-language-server
    chromium
    dig
    easytag
    # (import ./emacs-with-packages.nix {})
    ffmpeg
    fig2dev
    firefox
    gaw # analog signal viewer
    ghostscriptX # needed for fonts by xfig
    gimp
    git
    gnome-tweaks
    gnome-themes-extra
    gnomeExtensions.launch-new-instance
    gnomeExtensions.resource-monitor
    gnuplot
    gv
    inkscape
    jdk # bencrypt, ...
    libreoffice
    maxima
    mpv
    nix-prefetch-git
    pandoc
    pandoc-lua-filters
    pciutils
    pstoedit
    pyright
    scribus
    signal-desktop
    # texlive.combined.scheme-medium
    ((texlive.combine {
      inherit (texlive)  # space separated list of pkgs
        scheme-medium
        a4wide
        comment
        conv-xkv
        enumitem
        #esint     # closed surfaces integrals
        fontaxes  # required by svmono
        footmisc  # required by svmono
        framed    # required by svmono
        ieeetran
        multirow
        newtx     # required by svmono
        ntheorem  # required by svmono
        pagecolor
        pdfcolmk
        pgfplots
        standalone
        thmtools
        type1cm   # required by svmono
        xstring;  # required by svmono
      # Causes a "Too many open files" error. Need to set security.pam.loginLimits
      # pkgFilter = pkg:
      #   pkg.tlType == "run" || pkg.tlType == "bin" || pkg.tlType == "doc";
      pkgFilter = pkg: lib.elem pkg.tlType [ "run" "bin" "doc" ];
    }).override (args: { ignoreCollisions = true; }))
    thunderbird
    tree
    unison
    unzip
    vim
    # (import ./vscodium-with-extensions.nix pkgs)
    wget
    wxmaxima
    xfig
    xschem
    xyce
    yt-dlp
    zip
  ];

  ###########################################################333
  # GPG
  ###########################################################333
  
  # Some programs need SUID wrappers, can be configured further or are
  # started in user sessions.
  # programs.mtr.enable = true;
  programs.gnupg.agent = {
    enable = true;
    enableSSHSupport = true;
  };

  ###########################################################333
  # Misc
  ###########################################################333

  # TeXLive with doc needs to open more files than allowed by default
  security.pam.loginLimits = [{
    domain = "*";
    type = "soft";
    item = "nofile";
    value = "4096"; # four times the default;
  }];
      
  # Enable the OpenSSH daemon.
  services.openssh.enable = true;

  # Enable the LanguageTool serivice.
  services.languagetool = {
    enable = true;
    allowOrigin = "*"; # for Thunderbird add-on
  };
  
  # This value determines the NixOS release from which the default
  # settings for stateful data, like file locations and database versions
  # on your system were taken. It‘s perfectly fine and recommended to leave
  # this value at the release version of the first install of this system.
  # Before changing this value read the documentation for this option
  # (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
  system.stateVersion = "21.11"; # Did you read the comment?

}


qemu-kvm.nix:

{ config, pkgs, ... }:

{

  environment.systemPackages = with pkgs; [
    virt-manager
    virt-viewer
    spice spice-gtk
    spice-protocol
    virtio-win
    win-spice
    virtiofsd
  ];

  # Manage the virtualisation services
  virtualisation = {
    libvirtd = {
      enable = true;
      qemu = {
        swtpm.enable = true;
      };
    };
    spiceUSBRedirection.enable = true;
  };
  services.spice-vdagentd.enable = true;

}

hardware-configuration.nix:

{ config, lib, pkgs, modulesPath, ... }:

{
  imports =
    [ (modulesPath + "/installer/scan/not-detected.nix")
    ];

  boot.initrd.availableKernelModules = [ "xhci_pci" "ahci" "nvme" "usbhid" "sr_mod" ];
  boot.initrd.kernelModules = [ "dm-snapshot" ];
  boot.kernelModules = [ "kvm-intel" ];
  boot.extraModulePackages = [ ];

  fileSystems."/" =
    { device = "/dev/disk/by-uuid/...";
      fsType = "ext4";
    };

  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/...";
      fsType = "vfat";
    };

  swapDevices =
    [ { device = "/dev/disk/by-uuid/..."; }
    ];

  powerManagement.cpuFreqGovernor = lib.mkDefault "powersave";
  hardware.cpu.intel.updateMicrocode = lib.mkDefault config.hardware.enableRedistributableFirmware;
}

1 Like

Just for the sake of completeness, can you also provide the nixpkgs revision you used for the working boot entry and the non-working one?

Here the revision captured by /nix/var/nix/profiles/system-N-link/nixos-version.

The working one is 25.11.6561.1267bb4920d0, the non-working one is 25.11.7198.71caefce12ba.

1 Like

The problem seems to be that the kernel was not copied correctly, it’s size is 0 kB

$ cd /boot/EFI/nixos; ll
total 66546
-rwxr-xr-x 1 root root 27825400 Mar  2 14:10 0vz4i6ykbxky5nsr4qpq6ar16w6y3rvx-initrd-linux-6.12.74-initrd.efi
-rwxr-xr-x 1 root root 12493312 Mar  2 14:10 asbw2z87v1m06408vpkwwam0vvwwdr5z-linux-6.12.74-bzImage.efi
-rwxr-xr-x 1 root root        0 Mar  9 10:49 qxsp35yd75xjvxiyyqvj0lpxsx2q58gl-linux-6.12.75-bzImage.efi
-rwxr-xr-x 1 root root 27822822 Mar  9 10:49 sd4034ylmvprc0102nwy865i91ml00qj-initrd-linux-6.12.75-initrd.efi

The problem is definitely not partition space:

$ df -h .
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p1  911M  109M  802M  12% /boot

Re-running nixos-rebuild boot didn’t fix the problem, but deleting the empty file and re-running it did.

How is it possible that nixos-rebuild succeed, but there was a problem installing the boot kernel? Can we introduce a test in nixos-rebuild to make sure that the installed kernel is installed correctly (e.g., it is not empty) and fails if it is?

If it did fail, most likely the service failed and printed some kind of message that was swallowed up by the flake output truncation or something else along those lines.

What I’m more interested in is understanding why the boot entry was changed nonetheless. This will be a pretty extreme edge case, hard to know what happened without logs

Even with logs it might be tricky, quite possible this is a filesystem bug or power outage situation (fat doesn’t have a journal; kinda funny tbh that the partition most likely to cause boot failure uses the least fault tolerant file system). If you turn your systems off by holding power buttons or yanking power cables, this is your cue to stop.

I can add that there was no power outage, and that re-running nixos-rebuild boot didn’t rebuild anything and returned success. In spite of this, the 0 kB size kernel in the boot partition remained the same.

I can agree that this is an edge case. However, the kernel in the store was generated correctly. A simple check that the size of the kernel copied in the boot partition matches the one on the store would caught these kind of problems.

I’ve opened a bug report

1 Like