Diary of a new NixOS user - Thoughts/impressions/Obstacles

[TomB]

I thought I’d document my experience with NixOS as a new user. This is kind of a mini review and my first impressions.

As some background, I’ve been using Arch for about 10 years, I’ve used Mint and Fedora as daily drivers in the past and played around with Ubuntu, Gentoo and various other distros over the years. I’m not a new Linux user but never used NixOS. I’m very familiar with creating docker images and I have some experience with creating customised Arch PKGBUILD files though I’m not a maintainer so probably know just enough to be dangerous.

Day 1:

Grabbed the installer, wrote the ISO. For some reason the usb drive I was using didn’t boot in UEFI mode, swapped to another one and it worked as intended :shrug:.

I used the graphical installer as I wanted to go down the recommended path for installation. I was surprised that the installer didn’t give any filesystem options, it felt like a glaring omission but installed it using the installer and booted the OS. I didn’t expect ZFS or anything exotic to be an option but thought I’d at least be able to choose between ext4 and btrfs.

Installed, rebooted and it all worked fine, had a quick look around and explored the configuration.nix. file. But I’d previously read about impermanence (one of the things that drew me to nixos in the first place) so looked into how that worked. In addition I wanted disk encryption. Another feature that the installer surprisingly also doesn’t have an option for.

Firstly I wanted to see if I could get disk encryption and ZFS working. I followed the guide here: NixOS Root on ZFS — OpenZFS documentation

I can use cryptsetup to open the disk with a passphrase in the live usb but whatever I try, NixOS never asks me for the passphrase on boot after installation. It times out trying to mount the partition (obviously, it can’t see it until cryptsetup has opened the disk). It’s clearly not a ZFS issue, NixOS isn’t even aware it’s a ZFS partition until it’s opened with cryptsetup open. I tried various things suggested on the forum, including specifing the device using /dev/disk/by-id, /dev/disk/by-uuid, /dev/disk/by-partuuid etc. After about an hour of tinkering I gave up.

I tried again using ZFS encryption instead of luks2 as per this gist How I installed Encrypted ZFS root on NixOS · GitHub and Nix asks for the passphrase on boot first time. Not really sure of the pros/cons of this as I’ve only ever used luks2 in the past. I’ll leave it for now and investigate again later.

Installation 3, now I know I can get disk encryption and ZFS working I wanted to test out an impermanence configuration. I very easily got impermenance set up quickly using a mix of Erase your darlings: immutable infrastructure for mutable systems - Graham Christensen and NixOS ❄: tmpfs as home - Elis Hirwing and NixOS ❄: tmpfs as root - Elis Hirwing.

I took a hybrid approach where I’m using a ZFS partition for /nix and /nix/persist so I can leverage snapshots, mirroring and other nice ZFS features but using tmpfs for / and /home rather than reverting to a blank snapshot on boot. Still using ZFS encryption for now.

Day 1 - Initial Thoughts/Opinions

Having used both the graphical installer and the nixos-generate-config + nix-install commands, I strongly think that the graphical installer should say something like:

This installer is intended to make the installation process as simple as possible with minimal customisation.
If you’re familiar with manually creating partitions and linux in general, you may prefer to use the command line installer

nix-install is the easiest installer I’ve ever used. And doing it that way introduces the user to the configuration.nix file at the installation stage. In my opinion the GUI installer just gets in the way and is quite restrictive. Though I realise I am coming at this from the perspective of someone used to manually installing arch via the cli installer.

Nix’s declarative build system and immutable filesystem approach is surely the future of Linux. It’s a breath of fresh air for anyone who likes a clean and tidy system.

Impermanence, and how delightfully easy it is to set up, is definitely a killer feature here in my opinion. No more cluttered .local, .config and dozens of dotfiles in my home directory that I’m not sure if I need or not!

I wish I’d tried out Nix earlier! I’ll be using it as my daily driver at home to see how I get on.

The only annoyance I had was being unable to get luks2 to work, it wasn’t at all clear why it never asked for the password during boot.

[StepBroBD]

I’m currently using disko to format disks. After importing the disko module and your disko configs, disk(s) can be formatted with an one-liner:

bash $(nix --extra-experimental-features "nix-command flakes" build --no-link --print-out-paths github:<username>/<repo>#nixosConfigurations.<machine>.config.system.build.diskoScript)

If you want to use ZFS native encrypting with password prompt, be sure to check out this option

Some other interesting reading materials:

• Disko - NixOS Wiki
• doctor-cluster-config/modules/disko-zfs.nix at 46611f076724b28d13a2de0bf65625c16aa58bd9 · TUM-DSE/doctor-cluster-config · GitHub
• disko/example at master · nix-community/disko · GitHub
• Install NixOS with disko disk partitioning – NixOS Asia
• disko | foodogsquared's NixOS config
• disko and nixos-anywhere: Declarative and Remote Installation of NixOS :: NixCon 2023 :: pretalx
• dotfiles/systems/nixos/framework at master · stepbrobd/dotfiles · GitHub (ZFS native encryption with password prompt)

Have fun configuring!

[tasiaiso]

Thanks for the feedback! That’s pretty helpful to see what new users get stuck on.

@TomB

I used the graphical installer as I wanted to go down the recommended path for installation. I was surprised that the installer didn’t give any filesystem options, it felt like a glaring omission but installed it using the installer and booted the OS. I didn’t expect ZFS or anything exotic to be an option but thought I’d at least be able to choose between ext4 and btrfs.

Are you using the GNOME variant ? I’ve never used it, but the Plasma installer does let you mess with partitions (it uses Calamares). If this is the case, I do think we should make it clearer. As @StepBroBD said, disko is also an option, although I’ve never tried it either.

Thanks, keep us posted!

[Gaditone]

In addition I wanted disk encryption. Another feature that the installer surprisingly also doesn’t have an option for.

You sure?

[TomB]

@tasiaiso

Are you using the GNOME variant ? I’ve never used it, but the Plasma installer does let you mess with partitions (it uses Calamares). If this is the case, I do think we should make it clearer. As @StepBroBD said, disko is also an option, although I’ve never tried it either.

I was using Gnome, yes, though I’d have assumed that the installer was the same on both and only the DE was different.

It definitely let me resize them but didn’t let me chose an option.

@Gaditone

You sure?

I’ll have another look when I get a chance. Perhaps it’s because I chose the erase/full disk option during install, it’s entirely possible one of the other options offers different configuration settings.

Here’s a late Day 2 entry:

I normally use wayfire. Replaced gnome with programs.wayfire.enable = true; and I can’t launch it due to a permissions error. A forum post (which I now can’t find unfortunately) and this issue `services.xserver` naming is confusing · Issue #94799 · NixOS/nixpkgs · GitHub suggest that services.xserver.enable is needed for wayland. It makes no difference. I was trying to just execute wayfire from a tty like I can on other distros.

I check the documentation Wayland - NixOS Wiki which states: Two things are required for running Wayland: a compatible Display Manager, and a compatible Compositor. which struck me as odd because Wayland compositors normally act as the display manager but figured it might be a nuance of how NixOS works and tried lightdm, sddm and gdm. While the wayfire session was visible in both sddm and gdm it crashed on login.

I thought I’d try a different wlroots based desktop and installed sway. It launched immediately, even without a display manager from a tty. I thought it was strange, and retried wayfire. It launched. Removed sway from the config. Wayfire stopped working.

Created my first bug report on day 2 I realise Wayfire isn’t a common desktop environment so wasn’t entirely surprised that there was a bug. But I have a workaround, I just need to keep sway installed until it’s fixed.

Next step is to get my customised theme working and create the equivalent of my old hacky bash scripts that sed the theme files into a Nix package. Time to write my first package but that will have to wait until I have more time for tinkering.

Installed all the software I wanted without issue and day to day usage is stable and I’m enjoying NixOS so far. I’ll definitely be interested in trying it on my home server as well as my desktop with a slight reservation that it’s acting as a router and needs some complex networking configuration.

[jtojnar]

@TomB

I was using Gnome, yes, though I’d have assumed that the installer was the same on both and only the DE was different.

Yes, GNOME Live CD uses the same Calamares installer as Plasma variant.

The filesystem selection is only available under Manual partitioning. You can do all sorts of stuff with it but no ZFS and, at that point, I find using the CLI commands clearer. Also, creating a LVM volume using Calamares makes it crash for me – apparently, LVM supports is known to be buggy and it might be removed in the future.

I imagine we might eventually switch to Icicle, which has this done much more sensibly: If you go to Advanced partitioning, it will leave it up to you to create partitions. There is a button for launching GParted for convenience but you can use your favorite programs. Then you will be able to assign the mount points as you wish:

By the way, welcome to NixOS and best wishes from a former Dice user.