Help installing UEFI NixOS with root partition encryption as a second OS

Hi everybody,

I’ve been struggling with the successful installation of NixOS beside an already existing OpenSUSE 15.3. They are supposed to share /home, swap, and /boot/efi.

Following output is from the nixos installation USB

[nixos@nixos:/mnt]$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
loop0         7:0    0   1.3G  1 loop  /nix/.ro-store
sda           8:0    1  14.4G  0 disk  
├─sda1        8:1    1  14.4G  0 part  
│ └─ventoy  254:0    0   1.4G  1 dm    /iso
└─sda2        8:2    1    32M  0 part  
nvme0n1     259:0    0 931.5G  0 disk  
├─nvme0n1p1 259:1    0   512M  0 part  /boot/efi  #shared /boot/efi
├─nvme0n1p2 259:2    0  62.2G  0 part    # encrypted swap
├─nvme0n1p3 259:3    0   250G  0 part    # encrypted opensuse /home
├─nvme0n1p4 259:4    0   300G  0 part    # encrypted opensuse /
└─nvme0n1p5 259:5    0 318.8G  0 part   # encrypted NixOS installation
  └─cr_root 254:1    0 318.8G  0 crypt /mnt

/mnt/etc/nixos/configuration.nix

# Edit this configuration file to define what should be installed on
# your system.  Help is available in the configuration.nix(5) man page
# and in the NixOS manual (accessible by running ‘nixos-help’).

{ config, pkgs, ... }:

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
    ];

  # Use the systemd-boot EFI boot loader.
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;
  boot.loader.grub = {
    # efiInstallAsRemovable = true;
    enableCryptodisk = true;
    useOSProber = true;
  };
  boot.loader.efi.efiSysMountPoint = "/boot/efi";
  boot.initrd.preFailCommands = 
    "echo dev mapper\n" +
    "ls -l /dev/mapper\n" +
    "echo dev UUID\n" +
    "ls -l /dev/disk/by-uuid\n"
  ;

  # networking.hostName = "nixos"; # Define your hostname.
  # networking.wireless.enable = true;  # Enables wireless support via wpa_supplicant.

  # Set your time zone.
  # time.timeZone = "Europe/Amsterdam";

  # The global useDHCP flag is deprecated, therefore explicitly set to false here.
  # Per-interface useDHCP will be mandatory in the future, so this generated config
  # replicates the default behaviour.
  networking.useDHCP = false;
  networking.interfaces.enp2s0.useDHCP = true;
  networking.interfaces.wlo1.useDHCP = true;

  # Configure network proxy if necessary
  # networking.proxy.default = "http://user:password@proxy:port/";
  # networking.proxy.noProxy = "127.0.0.1,localhost,internal.domain";

  # Select internationalisation properties.
  # i18n.defaultLocale = "en_US.UTF-8";
  # console = {
  #   font = "Lat2-Terminus16";
  #   keyMap = "us";
  # };

  # Enable the X11 windowing system.
  services.xserver.enable = true;


  # Enable the Plasma 5 Desktop Environment.
  services.xserver.displayManager.sddm.enable = true;
  services.xserver.displayManager.autoLogin.enable = true;
  services.xserver.displayManager.autoLogin.user = "jojo";
  services.xserver.desktopManager.plasma5.enable = true;
  

  # Configure keymap in X11
  # services.xserver.layout = "us";
  # services.xserver.xkbOptions = "eurosign:e";

  # Enable CUPS to print documents.
  # services.printing.enable = true;

  # Enable sound.
  # sound.enable = true;
  # hardware.pulseaudio.enable = true;

  # Enable touchpad support (enabled default in most desktopManager).
  # services.xserver.libinput.enable = true;

  # Define a user account. Don't forget to set a password with ‘passwd’.
  users.users.jojo = {
    isNormalUser = true;
    home = "/home/jojo";
    extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user.
  };

  # List packages installed in system profile. To search, run:
  # $ nix search wget
  environment.systemPackages = with pkgs; [
    vim # Do not forget to add an editor to edit configuration.nix! The Nano editor is also installed by default.
    wget
    firefox
    chromium
  ];

  # Some programs need SUID wrappers, can be configured further or are
  # started in user sessions.
  # programs.mtr.enable = true;
  # programs.gnupg.agent = {
  #   enable = true;
  #   enableSSHSupport = true;
  # };

  # List services that you want to enable:

  # Enable the OpenSSH daemon.
  # services.openssh.enable = true;

  # Open ports in the firewall.
  # networking.firewall.allowedTCPPorts = [ ... ];
  # networking.firewall.allowedUDPPorts = [ ... ];
  # Or disable the firewall altogether.
  # networking.firewall.enable = false;

  # This value determines the NixOS release from which the default
  # settings for stateful data, like file locations and database versions
  # on your system were taken. It‘s perfectly fine and recommended to leave
  # this value at the release version of the first install of this system.
  # Before changing this value read the documentation for this option
  # (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
  system.stateVersion = "21.05"; # Did you read the comment?

}

/mnt/etc/nixos/hardware-configuration.nix

# Do not modify this file!  It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations.  Please make changes
# to /etc/nixos/configuration.nix instead.
{ config, lib, pkgs, modulesPath, ... }:

{
  imports =
    [ (modulesPath + "/installer/scan/not-detected.nix")
    ];

  boot.initrd.availableKernelModules = [ "nvme" "xhci_pci" "usb_storage" "sd_mod" "rtsx_pci_sdmmc" ];
  boot.initrd.kernelModules = [ ];
  boot.initrd.supportedFilesystems = [ "btrfs" ];
  boot.kernelModules = [ "kvm-amd" ];
  boot.extraModulePackages = [ ];

  # Root
  boot.initrd.luks.devices."cr_root".device = "/dev/disk/by-uuid/addf4189-1770-4575-ae52-4a46d51eebdf";
  fileSystems."/".device = "/dev/mapper/cr_root";
  #fileSystems."/".device = "/dev/disk/by-uuid/b862b21d-cf43-469f-95d7-ae51098bfb9f";

  fileSystems."/boot/efi/".device = "/dev/disk/by-uuid/6FB6-5F64" ;
  fileSystems."/boot/efi/".mountPoint = "/boot/efi";

  # Home
  boot.initrd.luks.devices."cr_home".device = "/dev/disk/by-uuid/69f8d66f-45d1-4047-8161-841dc704128a";
  fileSystems."/home".device = "/dev/mapper/cr_home";
  #fileSystems."/home".device = "/dev/disk/by-uuid/0175d2f8-aa4a-487c-bbc4-990624a59c1e";
  
  # Swap
  boot.initrd.luks.devices."cr_swap".device = "/dev/disk/by-uuid/863d9154-aeca-4e01-bc10-532cd2c59517";

  swapDevices = [
        { device = "/dev/mapper/cr_swap" ; }
        #{ device = "/dev/disk/by-uuid/ea99d871-f581-47ce-bb23-4065799cf803" ; }
  ];

}

Problems

This is the 5th or so iteration of trying to get this to work.

I was trying to fix something along the lines of mount /dev/mapper/cr_root on /mnt-root failed: no such file or directory. However, the output of boot.initrd.preFailCommands was showing that the partitions were being decrypted, but weren’t mounted under /mnt-root for some reason.

Now, it doesn’t get past

[nixos@nixos:/mnt]$ sudo nixos-install
building the configuration in /mnt/etc/nixos/configuration.nix...
/nix/store/bnbspvq0acxdnj55kwm5hxysflajnbrd-nixos-system-nixos-21.05.3862.8fe3b97ef45
copying channel...
installing the boot loader...
setting up /etc...
Failed to check file system type of "/boot/efi": No such file or directory
Traceback (most recent call last):
  File "/nix/store/aaxli88rvjhvf74xn545l1m2gdd65f40-systemd-boot", line 274, in <module>
    main()
  File "/nix/store/aaxli88rvjhvf74xn545l1m2gdd65f40-systemd-boot", line 212, in main
    subprocess.check_call(["/nix/store/n5j5fjn60nhck658j9ab84k8n9z24n1r-systemd-247.6/bin/bootctl", "--path=/boot/efi", "install"])
  File "/nix/store/32q6ryrb860sksdi1al5djg3pgcpq92l-python3-3.8.11/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/nix/store/n5j5fjn60nhck658j9ab84k8n9z24n1r-systemd-247.6/bin/bootctl', '--path=/boot/efi', 'install']' returned non-zero exit status 1.

As seen in lsblk /boot/efi is vfat and I can boot with OpenSuse into it:

[nixos@nixos:/mnt]$ /nix/store/n5j5fjn60nhck658j9ab84k8n9z24n1r-systemd-247.6/bin/bootctl --path=/boot/efi | cat
System:
     Firmware: n/a (n/a)
  Secure Boot: disabled
   Setup Mode: setup
 Boot into FW: supported

Current Boot Loader:
      Product: n/a
     Features: ✗ Boot counting
               ✗ Menu timeout control
               ✗ One-shot menu timeout control
               ✗ Default entry control
               ✗ One-shot entry control
               ✗ Support for XBOOTLDR partition
               ✗ Support for passing random seed to OS
               ✗ Boot loader sets ESP partition information
          ESP: n/a
         File: └─n/a

Random Seed:
 Passed to OS: no
 System Token: set
       Exists: yes

Available Boot Loaders on ESP:
          ESP: /boot/efi (/dev/disk/by-partuuid/46cfbc96-484e-4d4a-b891-113aa7849aeb)
         File: └─/EFI/systemd/systemd-bootx64.efi (systemd-boot 247)
         File: └─/EFI/BOOT/bootx64.efi (systemd-boot 247)

Boot Loaders Listed in EFI Variables:
        Title: opensuse-secureboot
           ID: 0x0000
       Status: active, boot-order
    Partition: /dev/disk/by-partuuid/46cfbc96-484e-4d4a-b891-113aa7849aeb
         File: └─/EFI/opensuse/shim.efi

        Title: openSUSE
           ID: 0x0002
       Status: active, boot-order
    Partition: /dev/disk/by-partuuid/46cfbc96-484e-4d4a-b891-113aa7849aeb
         File: └─/EFI/opensuse/grubx64.efi

        Title: Linux Boot Manager
           ID: 0x0003
       Status: active, boot-order
    Partition: /dev/disk/by-partuuid/46cfbc96-484e-4d4a-b891-113aa7849aeb
         File: └─/EFI/systemd/systemd-bootx64.efi

Boot Loader Entries:
        $BOOT: /boot/efi (/dev/disk/by-partuuid/46cfbc96-484e-4d4a-b891-113aa7849aeb)

Default Boot Loader Entry:
        title: NixOS (Generation 9 NixOS 21.05.3862.8fe3b97ef45, Linux Kernel 5.10.73, Built on 2021-10-22)
           id: nixos-generation-9.conf
       source: /boot/efi/loader/entries/nixos-generation-9.conf
      version: Generation 9 NixOS 21.05.3862.8fe3b97ef45, Linux Kernel 5.10.73, Built on 2021-10-22
   machine-id: 54f217fb66c049e1a6559accd6a6a824
        linux: /efi/nixos/f7531av858f46xaqgwik45053x5pbfa6-linux-5.10.73-bzImage.efi
       initrd: /efi/nixos/skwk6038nij8chp1wq65bhszg4zib94q-initrd-linux-5.10.73-initrd.efi
      options: init=/nix/store/w5ji8mry4j001swic5fv5cbf7hq6lshd-nixos-system-nixos-21.05.3862.8fe3b97ef45/init loglevel=4

Has anybody done something similar before and gotten it to work with NixOS?

Uhhh… well, this is awkward. Within seconds of posting this, I figured it out thanks to NixOS UEFI installation instructions incomplete · Issue #9096 · NixOS/nixpkgs · GitHub

sudo mkdir -p /mnt/boot/efi
sudo mount /dev/disk/by-uuid/6FB6-5F64 /mnt/boot/efi

And my mount /dev/mapper/cr_root on /mnt-root failed: no such file or directory problem was fixed by boot.initrd.supportedFilesystems = [ "btrfs" ]; . Can’t remember where I found that…

Only took me 3 months to install NixOS :sweat_smile: A graphical installer would be immensely appreciated.

There is some chatter about that here and there, and some people trying to start implementing it :slight_smile: Others feel it’s a bit pointless since you’ll start diving right into learning nix next, and you’ll get to enjoy chasing down similarly difficult stuff on that end before long.

See this thread for example if you’re interested: Why is there no "installer" for NixOS?

Either way, to set your expectations, currently NixOS does require a decent grasp of Linux sysadmin stuff, and even then has quite a learning curve. Hopefully as the community grows these things will become more polished, so welcome and good luck!