Nix iso unable to boot in UEFI mode (but other distros can)

I just made a PR for this issue. i found
loadfont ($root)/EFI/BOOT/unicode.pf2 can not be used for detection. i will always pass.
loadfont ($root)/EFI/BOOT/grub-theme/dejavu.pf2 check will not pass for my testingpc, so i choose this for detection code

Yeah, as you might know from recent convos here and there, I have lots of thoughts, especially after having been through the whole ordeal of trying to produce install media (whether iso or a disk image) for ARM systems and finding out about the bespoke grub iso config, lack of DT support (now available in systemd-boot though) etc.

However, I canā€™t sign up for that, and if I were, Iā€™d probably take a fairly different approach and try to get lanzaboote and bootspec to be usable for installing a bootloader outside of the nixos configuration itself. This would lead to easier cross-arch installs, a unified systemd-boot approach with ā€œour ownā€ stub, etc. (Imagine a lanzaboote-nixos-{generate,install} that can take a system closure, read bootspec, assembly an image/iso, and/or do a cross-arch install by install the EFI binaries, creating the -system symlink, etc.)

Iā€™d love to brainstorm that, and could even potentially sling some Rust, but I canā€™t drive it. Iā€™m also hoping @21paradox is going to come up with an awesome patch for the short-term. :slight_smile: Thank you so much for looking into this @21paradox.

1 Like

Iā€™m happy to help test the PR on my unit and provide my results. Please share the URL and let me know how I can help.

you can check this pr nixos/installer/cd-dvd: loadfont check before gfxterm init, to prevent crash in nonuefi graphic card platform by 21paradox Ā· Pull Request #345984 Ā· NixOS/nixpkgs Ā· GitHub

2 Likes

Iā€™ve started my tests and so far it doesnā€™t work both in graphical and text mode. Iā€™ve never debugged UEFI / pre stage 1 (any tips would be welcome)

For now Iā€™m trying to see whatā€™s different in an ISO that worksā€¦currently comparing against https://download.opensuse.org/tumbleweed/iso/openSUSE-Tumbleweed-DVD-x86_64-Current.iso

@mastoca, Presumably youā€™d want/need to be testing with an ISO built with @21paradoxā€™s proposed changes. Can you confirm what ISO youā€™re testing with?

trying to keep it as simple as possible I did the following:

{ config, pkgs, ... }:
{
  imports = [
    ./nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix

    # Provide an initial copy of the NixOS channel so that the user
    # doesn't need to run "nix-channel --update" first.
    ./nixpkgs/nixos/modules/installer/cd-dvd/channel.nix
  ];
  environment.systemPackages = [ pkgs.neovim ];
}

built with
nix-build './nixpkgs/nixos' -A config.system.build.isoImage -I nixos-config=iso.nix

wrote to iso with

sudo dd of=/dev/sda bs=4M conv=fsync status=progress if=result/iso/nixos-24.11pre-git-x86_64-linux.iso

build output:

these 31 derivations will be built:
  /nix/store/5wvjpxbyrrwjb0cr3dj46brzxw9j70zm-lazy-options.json.drv
  /nix/store/xprr6i92wkckm1wwzf6a6hlghwzq4pln-options.json.drv
  /nix/store/0mrp22rh2zp2qjv1ywnflqndj4ixqddx-nixos-manual-html.drv
  /nix/store/vmbwcm2674bx6gypv3zhc39gfvj6a8w9-nixos-help.drv
  /nix/store/alwiydx1l7c5j1iicmfgzbbkcnrk5khj-nixos-help.drv
  /nix/store/yy017asyxpkpvfhgsx0qi6rsl5dnk2yy-nixos-configuration-reference-manpage.drv
  /nix/store/1fxmq38rrbqfi0ahd25kijxhq9xfxagi-system-path.drv
  /nix/store/4vx8abjyfpjj51qfnfa4csj6z0ylmdwx-nixos-24.11pre-git.drv
  /nix/store/2n0g754pmha4hx4a6hlx60c38wa47d37-etc-nix-registry.json.drv
  /nix/store/h5d7mgcym1jndi2fvp0yzy8p8528i6yf-X-Restart-Triggers-polkit.drv
  /nix/store/6hcflyxhnbgvnw411a10h9xmglwfbfyv-unit-polkit.service.drv
  /nix/store/6zab4b9h49wyx66l5k40llcan8gz320r-unit-accounts-daemon.service.drv
  /nix/store/gklxxr3b9pj0cwhwhn4j75arbji9gscc-dbus-1.drv
  /nix/store/ghl7k5cp29i1rh231ismvxakvb99gi13-X-Restart-Triggers-dbus.drv
  /nix/store/hgq917phca32shb187xc9l8y5x8va2qx-unit-dbus.service.drv
  /nix/store/3j4ancf3zc2jpl3g70qjn87g1vxwq4ap-system-units.drv
  /nix/store/9y9hn2mhh8q02v8wjmhvr5w7kwnhjrwh-set-environment.drv
  /nix/store/499rzqmxl0vwj6j55l2dh1z76li7illr-etc-profile.drv
  /nix/store/d9sf75k3smk1znkqjlifvik6x68xhddi-etc-pam-environment.drv
  /nix/store/yn5k53ykzh09a6ds7y5hh14jv8k0nakw-unit-dbus.service.drv
  /nix/store/v6maylv34m3d9mwrb6hrgrry7vyw7pap-user-units.drv
  /nix/store/9z562hppdrm76bjd1mfa8rd1dq6wfk15-etc.drv
  /nix/store/2kn841k4p0166ml0n3zpvazicyhrp4s7-local-cmds.drv
  /nix/store/ddcbv8r7jmwxx8q3y8l4801k04g7kv8z-stage-2-init.sh.drv
  /nix/store/pr40a90z6s9z6g9zxmgbz0psc0znjrgd-nixos-system-nixos-24.11pre-git.drv
  /nix/store/0pm582p29m37hgxslbwblgdr997203hd-closure-info.drv
  /nix/store/dc2m9z2l30m2x9l41a1sdl6qf8pz7ya0-isolinux.cfg-in.drv
  /nix/store/3z6ihg79pwca0bdscy7jsp4rm77wf77s-isolinux.cfg.drv
  /nix/store/yarfai2chzfl6a7527q6091lklmvb8qc-efi-directory.drv
  /nix/store/vxh6c2bgbyqnc38hgv0xrcpzvzq6lqyq-efi-image_eltorito.drv
  /nix/store/6f20yv4xhg6z92b5nyzlhrp5gnsxnk3h-nixos-24.11pre-git-x86_64-linux.iso.drv
building '/nix/store/4vx8abjyfpjj51qfnfa4csj6z0ylmdwx-nixos-24.11pre-git.drv'...
building '/nix/store/5wvjpxbyrrwjb0cr3dj46brzxw9j70zm-lazy-options.json.drv'...
building '/nix/store/xprr6i92wkckm1wwzf6a6hlghwzq4pln-options.json.drv'...
building '/nix/store/yy017asyxpkpvfhgsx0qi6rsl5dnk2yy-nixos-configuration-reference-manpage.drv'...
building '/nix/store/0mrp22rh2zp2qjv1ywnflqndj4ixqddx-nixos-manual-html.drv'...
building '/nix/store/vmbwcm2674bx6gypv3zhc39gfvj6a8w9-nixos-help.drv'...
building '/nix/store/alwiydx1l7c5j1iicmfgzbbkcnrk5khj-nixos-help.drv'...
building '/nix/store/1fxmq38rrbqfi0ahd25kijxhq9xfxagi-system-path.drv'...
created 17501 symlinks in user environment
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
building '/nix/store/h5d7mgcym1jndi2fvp0yzy8p8528i6yf-X-Restart-Triggers-polkit.drv'...
building '/nix/store/gklxxr3b9pj0cwhwhn4j75arbji9gscc-dbus-1.drv'...
building '/nix/store/d9sf75k3smk1znkqjlifvik6x68xhddi-etc-pam-environment.drv'...
building '/nix/store/9y9hn2mhh8q02v8wjmhvr5w7kwnhjrwh-set-environment.drv'...
building '/nix/store/6zab4b9h49wyx66l5k40llcan8gz320r-unit-accounts-daemon.service.drv'...
building '/nix/store/ghl7k5cp29i1rh231ismvxakvb99gi13-X-Restart-Triggers-dbus.drv'...
building '/nix/store/2n0g754pmha4hx4a6hlx60c38wa47d37-etc-nix-registry.json.drv'...
building '/nix/store/499rzqmxl0vwj6j55l2dh1z76li7illr-etc-profile.drv'...
building '/nix/store/2kn841k4p0166ml0n3zpvazicyhrp4s7-local-cmds.drv'...
building '/nix/store/6hcflyxhnbgvnw411a10h9xmglwfbfyv-unit-polkit.service.drv'...
building '/nix/store/ddcbv8r7jmwxx8q3y8l4801k04g7kv8z-stage-2-init.sh.drv'...
building '/nix/store/hgq917phca32shb187xc9l8y5x8va2qx-unit-dbus.service.drv'...
building '/nix/store/yn5k53ykzh09a6ds7y5hh14jv8k0nakw-unit-dbus.service.drv'...
building '/nix/store/3j4ancf3zc2jpl3g70qjn87g1vxwq4ap-system-units.drv'...
building '/nix/store/v6maylv34m3d9mwrb6hrgrry7vyw7pap-user-units.drv'...
building '/nix/store/9z562hppdrm76bjd1mfa8rd1dq6wfk15-etc.drv'...
building '/nix/store/pr40a90z6s9z6g9zxmgbz0psc0znjrgd-nixos-system-nixos-24.11pre-git.drv'...
building '/nix/store/0pm582p29m37hgxslbwblgdr997203hd-closure-info.drv'...
building '/nix/store/yarfai2chzfl6a7527q6091lklmvb8qc-efi-directory.drv'...
building '/nix/store/dc2m9z2l30m2x9l41a1sdl6qf8pz7ya0-isolinux.cfg-in.drv'...
structuredAttrs is enabled
Building GRUB with modules:
 - fat
 - iso9660
 - part_gpt
 - part_msdos
 - normal
 - boot
 - linux
 - configfile
 - loopback
 - chain
 - halt
 - efifwsetup
 - efi_gop
 - efi_uga
 - ls
 - search
 - search_label
 - search_fs_uuid
 - search_fs_file
 - echo
 - serial
 - gfxmenu
 - gfxterm
 - gfxterm_background
 - gfxterm_menu
 - test
 - loadenv
 - all_video
 - videoinfo
 - png
Adding additional modules:
 - efi_uga
'/nix/store/pf7m17iz3w1597kh314xqk6lfwz3r7vf-refind-0.14.2/share/refind/refind_x64.efi' -> '/nix/store/58glr6rz3ycahw5zr2k6cy86qxhvmhqc-efi-directory/EFI/BOOT/refind_x64.efi'
building '/nix/store/vxh6c2bgbyqnc38hgv0xrcpzvzq6lqyq-efi-image_eltorito.drv'...
building '/nix/store/3z6ihg79pwca0bdscy7jsp4rm77wf77s-isolinux.cfg.drv'...
Usage size: 2097152
Image size: 3145728
mkfs.fat 4.2 (2021-01-31)
Copying BOOTX64.EFI
Copying grub.cfg
Copying refind_x64.efi
fsck.fat 4.2 (2021-01-31)
Checking we can access the last sector of the filesystem
Boot sector contents:
System ID "mkfs.fat"
Media byte 0xf8 (hard disk)
       512 bytes per logical sector
      2048 bytes per cluster
         1 reserved sector
First FAT starts at byte 512 (sector 1)
         2 FATs, 12 bit entries
      2560 bytes per FAT (= 5 sectors)
Root directory starts at byte 5632 (sector 11)
       512 root directory entries
Data area starts at byte 22016 (sector 43)
      1525 data clusters (3123200 bytes)
32 sectors/track, 2 heads
         0 hidden sectors
      6144 sectors total
Checking for unused clusters.
/nix/store/cp4wxsv4p70b3v4krlh2bygckq2rc117-efi-image_eltorito: 6 files, 708/1525 clusters
building '/nix/store/6f20yv4xhg6z92b5nyzlhrp5gnsxnk3h-nixos-24.11pre-git-x86_64-linux.iso.drv'...
structuredAttrs is enabled
Parallel mksquashfs: Using 32 processors
Creating 4.0 filesystem on nix-store.squashfs, block size 1048576.


Exportable Squashfs 4.0 filesystem, zstd compressed, data block size 1048576
        compressed data, compressed metadata, compressed fragments,
        compressed xattrs, compressed ids
        4k unaligned
        duplicates are removed
Filesystem size 2229862.16 Kbytes (2177.60 Mbytes)
        36.54% of uncompressed filesystem size (6103164.53 Kbytes)
Inode table size 2414922 bytes (2358.32 Kbytes)
        20.99% of uncompressed inode table size (11504361 bytes)
Directory table size 2414462 bytes (2357.87 Kbytes)
        34.67% of uncompressed directory table size (6964074 bytes)
Number of duplicate files found 11331
Number of inodes 274644
Number of files 162487
Number of fragments 2727
Number of symbolic links 41569
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 70588
Number of hard-links 0
Number of ids (unique uids + gids) 1
Number of uids 1
        root (0)
Number of gids 1
        root (0)
xorriso 1.5.6 : RockRidge filesystem manipulator, libburnia project.

xorriso : NOTE : Environment variable SOURCE_DATE_EPOCH encountered with value 315532800
Drive current: -outdev 'stdio:/nix/store/qiwd3h4djp6c53jldb4hzspmzj4fib46-nixos-24.11pre-git-x86_64-linux.iso/iso/nixos-24.11pre-git-x86_64-linux.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data,  161g free
xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
xorriso : UPDATE :     195 files added in 1 seconds
Added 13 items from file 'pathlist'
xorriso : UPDATE :     195 files added in 1 seconds
xorriso : NOTE : Copying to System Area: 432 bytes from file '/nix/store/0g248phyddy23jsggl4ciz3dpg3d3hj3-syslinux-unstable-2019-02-07/share/syslinux/isohdpfx.bin'
libisofs: NOTE : Automatically adjusted MBR geometry to 1020/140/32
libisofs: NOTE : Aligned image size to cylinder size by 198 blocks
xorriso : UPDATE :  2.25% done
xorriso : UPDATE :  19.43% done
xorriso : UPDATE :  42.25% done
xorriso : UPDATE :  67.46% done, estimate finish Fri Oct 04 16:42:27 2024
xorriso : UPDATE :  90.90% done
ISO image produced: 1142400 sectors
Written to medium : 1142400 sectors at LBA 0
Writing to 'stdio:/nix/store/qiwd3h4djp6c53jldb4hzspmzj4fib46-nixos-24.11pre-git-x86_64-linux.iso/iso/nixos-24.11pre-git-x86_64-linux.iso' completed successfully.

/nix/store/qiwd3h4djp6c53jldb4hzspmzj4fib46-nixos-24.11pre-git-x86_64-linux.iso
1 Like

Nice. Not that itā€™s bad here, but your feedback about it not working, or maybe being an insufficient fix, would be useful on the PR itself, as well. (along with repeating any details you might have about hardware, or suspicions).

(Might also be good to cross-link to Discourse when you reply)

Iā€™m so glad yā€™all are PR/testing this, especially since itā€™s maybe been broken semi-quietly for some time!

2 Likes

thanks, good suggestion, Iā€™ll chime in on the PRā€¦ it is frustrating that debug option doesnā€™t show anything that could tell me where itā€™s tanking.

you can use rufus, write to usb stick with iso mode,then you can edit files in that usb stick

1 Like

Thank you I have a test loop now that will let me explore what is going on. So far, Iā€™ve not got to Stage 1 yet but did manage to at least cause a different failure (on purpose). Iā€™ll explore some more and report as I make progressā€¦but this is a slow process since the console output is being sent elsewhere I canā€™t see after our kernel starts to load.

you can try adding forceTextMode=true, and build again. that will use the minimal default setting for iso boot

environment.systemPackages = [ pkgs.neovim ];
isoImage.forceTextMode = true; // force use textmode

Quick reportā€¦it is still no go with the text option for the minisforum.

Iā€™ve been looking at SUSEā€™s tumbleweed installer (it works) which uses kernel 6.9 so I copied their linux file to the installer (and renamed to bzImage etc.) I managed to boot into the SUSE kernel and by hand I got the squashfs mounted and started constructing the nixos envā€¦which was partially workingā€¦though I donā€™t remember all the steps of a rebuild switch. It was a fun rabbit hole and Iā€™m learning a ton.

My gaze is now on investigating whatā€™s wrong with our kernel (tried latest and 6.6.x). To that end Iā€™m currently building the kernel (6.10) following Linux kernel - NixOS Wiki to add debugging.

Iā€™ve not done kernel debugging in many many many years so we shall see what neurons in my mind wake up to chip in.

latest iso.nix looks like this for the curious.

{
  imports = [
    ./nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix

    # Provide an initial copy of the NixOS channel so that the user
    # doesn't need to run "nix-channel --update" first.
    #./nixpkgs/nixos/modules/installer/cd-dvd/channel.nix
  ];
  environment.systemPackages = [ pkgs.neovim ];
  boot.kernelPackages = pkgs.linuxPackages_6_10;
  boot.crashDump.enable = true;
  boot.kernelPatches = [ {
  name = "crashdump-config";
  patch = null;
  extraConfig = ''
          CRASH_DUMP y
          DEBUG_INFO y
          PROC_VMCORE y
          LOCKUP_DETECTOR y
          HARDLOCKUP_DETECTOR y
        '';
  } ];
  isoImage.forceTextMode = true;
}

Sadly I donā€™t think we are having any crash but just a bad state with no ui or tty.

1 Like