Install report from new user

Linux user since ~1995.

Migrating from Gentoo, after yet another portage package database screwup would mean reinstalling.

I am a big fan of functional/declarative, so was really keen to check out NixOS. Also using ARM64 at work for embedded and looking for a deployment solution that doesn’t suck, so the fact there’s some interest in ARM support already is great.

First I tried to download the iso but I couldn’t get it from here in China, although it’s working today. I guess the Amazon S3 IP that was assigned on that day for that release was Chinese firewall blocked for some reason. Got it downloaded very slowly via VPN.

So, situation before beginning: I already had a ZFS formatted SSD in my Dell XPS15. However, I was unable to install to it from the official install media because there is apparently no kernel support, userspace tools or documentation around installing directly to root on ZFS… only installing first to root on another filesystem, then adding ZFS support. (Update: This is allegedly a false assumption! Read onward to understand.)

After realising this, finally I saw it was necessary to wipe my disk. I spent a lot of time backing up files I am yet to restore. I then tried installing with GPT but couldn’t get grub2 to install for love nor money. Finally I used a legacy DOS MBR and got it to install.

No wireless. Reboot, add wireless, reboot, figure out NixOS way to wrap wpa_supplicant config, reboot again, working.

As a first time user, I can’t say this was a great install experience even though I have a pretty high tolerance for hassle being a Gentoo refugee.

However, once installed - WOW! So fast, nearly everything just works so far.

Random feedback:

  • Default CJK font rendering in Firefox is terrible, having better default fonts could be a good idea.
  • Yet to see any sound working
  • Resuming after suspend seems to not work with lid opening, instead requiring pressing the power button. OS then hangs. I generally never allocate swap space but did on this install hoping suspend/resume would ‘just work’ … no such luck
  • I would like to understand more about NixOS recommended strategies for filesystem layout for desktop use.
  • A graphical installer as discussed in other threads would definitely grow the community extremely quickly.
  • Would prefer auto login for X11 as a non-root user but unsure how to set that up
  • Tried to get a boot logo happening but couldn’t figure out how to make nixos-rebuild see the .png file in question
  • I will probably visit an ARM board manufacturer in the next few weeks here in China, Firefly, so if anyone wants hardware let me know. I could ask them to sponsor NixOS and/or Gentoo and get some free or discount boards sent out.
7 Likes

Right, it’s difficult when you miss kernel support for something you want in the first installation step. You’d need some knowledge and running nix to create a custom installation ISO.

I have at least four different HW with NixOS, and all of them have working suspend-resume (to RAM) out of the box, without any issues. (I use it a lot.) I suppose it depends on particular HW/drivers.

See docs for services.xserver.displayManager.*.autoLogin, e.g. in man configuration.nix, nixos-help or NixOS 23.11 manual | Nix & NixOS

1 Like

Hy,
I’ve follow a similar path around 6 months ago :slight_smile:

On zfs I do not exactly understand why you can’t reuse your pool,
of course you have to create a new root volume but that’s all.
Incompatible pools may come from different zfs distribution, for
instance IME FreeBSD and OpenSolaris/IllumOS pools are not importable
in GNU/Linux LLNL zfs port, also if for some reason your pool is
from a newer zfs version you can’t import but NixOS zfs is fairly
updated and LLNL port does not advance that much…

If you have problems in automounting volumes with systemd because
they are managed by zfs instead of “legacy” it’s only a matter to
change mount settings withto legacy “zfs set mountpoint=legacy pool/vol”.

A small side-note, to restore zfs consider using mbuffers instead
of netcat/ssh send&recv it’s far more faster.

About swap if you have it in a zvol that maybe the source of crash
on suspend due to zfs memory management, I think you need a separate
swap partition, inside LUKS if you want, but outside zpool…

On Grub2 yes, encountered the very same problem, simply solved with

  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;
  boot.loader.efi.efiSysMountPoint = "/boot/EFI";

Witch means use Systemd bootloader instead of Grub since it’s broken
on EFI install on NixOS

– Ingmar

1 Like

@grahamc had a positive experience with NixOS on Dell 9560, described in his blog post.

Regarding the ZFS, you would have to create another installation ISO supporting ZFS (there are creation instructions in the manual - here).

I am glad that NixOS worked well for you!

Some random comments:

  • For auto-login have a look at services.xserver.displayManager.lightdm.autoLogin.enable or anything equivalent.
  • For booting logo, you could use plymouth, e.g. boot.plymouth.enable = true;

As for the Firefly devices, let me know if you can send stuff to Europe and how it would cost, you got my attention :smiley:

1 Like

We had zfs in past in the default installer, but removed it due some people having concerns regarding the license. Realistically I don’t see any company soon to sue us for that. I guess somebody could sneak in an iso image automatically build on travis-ci or so and hope for the best. I would always recommend with zfsonlinux and systemd to use the legacy mounts. Automounts are probablematic until we have a way to inform systemd that orders all the mounts to be aware of the zfs pools.

1 Like

We had zfs in past in the default installer, but removed it due some
people having concerns regarding the license. Realistically I don’t
see any company soon to sue us for that. I guess somebody could sneak
in an iso image automatically build on travis-ci or so and hope for
the best.
Why not instead offer a quick way to produce personal ISOs within the
live image? My very first NixOS install was on zfs root and I built
the iso within the live now is http://ix.io/1rf4 for zfs suffice to
substitute nilfs2 with zfs. To build the iso

nix-build '<nixpkgs/nixos>' -A config.system.build.isoImage -I
nixos-config=iso.nix

Only limitation is amount of ram needed… Perhaps live image can offer
alongside with quick docs few nix module to generate various type of
ISO, like with zfs, nilfs2, f2fs, various hw quirks, … Users can
simply “merge” them as they like. As another companion a tarball of a
minimal chrootable NixOS root can be offered to produce custom iso
directly from another distro. It will be a bit big but not much bigger
than a desktop iso and since it produce the new iso locally, eventually
downloading needed stuff it’s not much different than classic dd-ing of
an iso. To exaggerate it can also have a small curses wrapper like
Systemrescuecd iso-installer :slight_smile:

– Ingmar

We had zfs in past in the default installer, but removed it due some
people having concerns regarding the license. Realistically I don’t
see any company soon to sue us for that. I guess somebody could sneak
in an iso image automatically build on travis-ci or so and hope for
the best.
Why not instead offer a quick way to produce personal ISOs within the
live image? My very first NixOS install was on zfs root and I built
the iso within the live now is http://ix.io/1rf4 for zfs suffice to
substitute nilfs2 with zfs. To build the iso

Isn’t it enough to install the modules then modprobe them?

There are some other cases (Nvidia, I guess) where a reboot is indeed
needed; I wonder if some kexec trick could be made to work to reboot
but keep an overlay with store changes…

Default CJK font rendering in Firefox is terrible, having better default fonts could be a good idea.

I agree that NixOS could really use some better defaults for CJK fonts.

I wrote up a small explanation of how to get nice-looking Japanese fonts setup on NixOS. I bet you could easily adopt it for Chinese fonts as well:

4 Likes

I’d write more but I’m traveling and in my phone. You don’t need to make your own ISO for ZFS support. You can enable it in the existing ISO at run time. See: https://nixos.wiki/wiki/NixOS_on_ZFS

2 Likes

Yeah, that blog post is a classic for CJK users. Personally, I’ve had some good experiences with the Noto font family (noto-fonts, noto-fonts-cjk[, noto-fonts-emoji for ☆ etc.]).

2 Likes

Hy,
I never tried sincerely, when I start NixOS I have used zfs but I’m
already exploring to abandon it since, unfortunately, it’s a dead
project for me… In theory you can compile zfs with dkms so load it
without reboot but I never tried. Espetially since produce a custom
and uo-to-date NixOS iso i so simple :slight_smile:

– Ingmar

1 Like

Hi everyone, thanks for the welcoming, polite and useful feedback. I will try all the suggestions over the weekend.

@joko I have just confirmed visiting the ARM board manufacturer and meeting their firmware and management teams next Thursday, I will let you know here afterwards.

Interesting. This wasn’t immediately obvious coming from another distribution (Gentoo) where the install media is not equivalent to the post-install environment with respect to kernel module installation and package management. I have added a prominent note to the bottom of the ZFS wiki page.

1 Like

Thanks, I have integrated some of your suggestions to my local configuration. The quality of inline documentation is excellent. Unfortunately I can’t seem to determine how to list the valid values for i18n.inputMethod.fcitx.engines and obvious values based on the fcitx website like pinyin and fuzzypinyin don’t seem to work.

Update: I later found some related source code but couldn’t figure out where the output was generated. Later found the awesome configuration.nix man page and determined that was the place. Voila! NixOS is consistently impressing me with its combination of clarity, tradition and precision with an overarching sense of progressive pragmatism. Great stuff!

For future users stumbling upon this post: man nixos.configuration then press / (forward slash) to search, and type fcitx.engines. You will then be presented with awesome documentation. This works for all configuration.nix options.

1 Like

Thanks. I added this information to the ZFS wiki page to help future users.

Thanks! That at least seemed to suggest one possible reason I may be experiencing crashes resuming, a BIOS option related to Wake on USB. Will check that out.

I have added this information as well as @grahamc 's reply and @Mic92 's background to the ZFS wiki page.

Also @joko @dezgeg let me know if you have any specific questions to put to Firefly, the RK3399 board manufacturer and I will make sure they are answered.

It turned out I had no idea you could simply install ZFS on the environment the installation media provides without rebooting. According to @grahamc this is possible. I have not tried it. Because I did not have that knowledge, I had no way to get ZFS running and therefore no way to access the existing pool from the installation media, which was the reason for backup and replacement. As it turned out, that wasn’t so painful and maybe was a good idea (I skipped world backup day this year…). Thanks also for your performance and other tips. Your grub2 info is great, I would have made use of it in the installation if I could have seen such a snippet. Would it be worth integrating in to the installation section of the manuall?

Hay walter,
being not on GitHub (nor wanting to be, I avoid any proprietary platform
I can avoid) I do not know how to contribute to the wiki… Anyway for
you remember nixos is functional so you can even change bootloader in
you config in a snap, simply write the snippet and run
nixos-rebuild switch and new bootloader will be installed. Any change
you made in the config can be easily applied and reverted as long as
you keep a working config around.

Even if you system became unbootable you can boot the live image and
re-run nixos-install with your last fully working config and have the
system restored without touching your home. Of course proper use of zfs
snapshots and clone may also help (poor man’s OpenSolaris/IllumOS BE),
the sole world of warning I’ve encountered: on install from the live
image /nix/store used to build final system is not placed in the final
root volume (e.g. something like /mnt/nix/store) but in live image
ram-mounted /nix/store so if you have a big config and a not so big ram
you may exhausts ram and being unable to finish install. In that case it
only suffice to comment out a (big) bunch of packages and after booting
in the new system uncomment them back and nixos-rebuild switch.

Also for upgrading NixOS, root volume should be considered around 4 time
the size of a classic root volume on Gentoo or other classic distro,
simply because when you upgrade from a release to another or made big
change or simply upgrade you do not really overwrite anything but only
download and extract/build anything aside the actual “root” resulting in
a big space occupation even if “temporary”. In case of need you can
garbage-collect by hand of course. My personal desktop recipe (in
configuration.nix) is

 # this is a system-wide settings for ANY shell you use (bash, zsh,
 # tcsh, ...) available to ALL users, root included.
  environment.shellAliases = {
    sysup = "sudo nixos-rebuild switch --upgrade && nix-env -u";
    sysclean = "sudo nix-collect-garbage -d ; sudo nix-store --gc; sudo nix-store  --optimize";
    sysbe = "sudo nix-env -p /nix/var/nix/profiles/system --list-generations";
  };

Being in the (far) past a Gentoo user I know how sometime a rebuild
world can be a bit scared, but on NixOS you do not have to be worried,
nothing can really brake to a “pain”/“long” point. Being “brave” it’s
safe enough.

– Ingmar

1 Like

Thanks Ingmar. Interesting aliases, thanks for sharing. As I mentioned in the original post I wanted to understand what the recommended filesystem structure was … it seems that in fact to make the most use of Nix one should have /home/* and /root/ on different volumes to the root partition.

As this is not explained in the install instructions, it makes modifying the current config post-facto difficult if you do not have another disk around and time for hoop-jumping (I don’t). Therefore, I will not be adventuring around boot loaders again any time soon. Very happy to have a functional system which is so fast and neat!

I also try to gravitate toward free software but I am very time constrained and sometimes there are no viable alternatives, particularly when issues like community, reputation, geographic replication, etc. are at stake and you have limited time and resources to devote. Frankly if I fervently insisted on open source at this point I wouldn’t be able to run my business. After years of running my own svn, git, gitlab and phabricator servers, CI/CD pipelines, private clusters, etc. I am now a paying enterprise customer of Github and very happy. Same for email (I avoid Google though!). I have run this stuff myself for 20+ years but simply have to give up as the bottleneck is now time, and I have enough issues to deal with between work, family, health, banks, government, etc. to try to pull it all together 24x7x365 when many times the annual service fees of these businesses are at stake every day.

Indeed, I can only find time for this type of discussion at 8:30PM on a Sunday. Must be getting old. :slight_smile:

Best, Walter

1 Like

Hi,

I wanted to understand what the recommended filesystem structure was
… it seems that in fact to make the most use of Nix one should have
/home/* and /root/ on different volumes to the root partition.
well, there is probably no “guide” because anyone decide to slice disks
so data as per personal need and taste. Normally having a separate home
volume means have their personal data separate from the system, like in
a car you have engine separated from cabin, this make easy in case of
snapshot-enabled fs/storage solution to backup the home, to modify the
system (i.e. change distro) without touch personal data etc.

My personal single-disk desktop recipe is:

  • EFI partition
  • boot partition separated because of LUKS encryption
  • LUKS volume with zpool on top in the past, lvm now
  • root
  • var
  • varhome
  • dothome
  • datahome

where var is separate to have logs and various spool separated from
the root, sometime I may would like to keep them while erasing the
system root, varhome is the /home/$myuser or the container of anything
I do not care (cache, defaults apps settings etc), dothome are my
dotfiles symlinked (in the past by hand, after with GNU stow, now I
try Nix-integrated homeManager) in /home/$myuser, datahome are the
rest of my personal data (documents, projects, images, videos, music,
…). When I have used zfs due to it’s flexible volume size I have
had many more volumes for home, one for my personal kb/wiki, another
for projects, another for docs, another for multimedia stuff etc and
for system (petc, for custom etc configs, linked in /etc like home
dotfiles).

Consider your scenario: you switch from Gentoo to NixOS and you have
both on zfs. If your home is on a dedicated volume you can simply
create a rpool/nixos volume aside rpool/gentoo volume and simply
change bootloader config. Your home remain there (backed up for safety
but untouched). Far simpler that have to backup and restore everything.
However you may encounter few problem with modern DE like Gnome/Kde
because they may badly digest you ancient DE dotfiles. So having only
the setting you care separated and an “erasable” home for other/defaults
setting may help migration, you simply wipe the “erasable home” run the
new DE and re-link back your relevant configs.

Now add a bit of complexity: you may have many different desktops for
different kind of work, for instance a personal laptop, a work laptop,
a personal desktop and a work desktop. You may want to share some data
between them, but not all. If you have different volumes for them it
may be only a matter of zfs send few volumes, or you may use unison,
my actual preferred way, for two way sync of ONLY the tree you want
too keep in sync. Here again slicing and (sym)linking/bind-mount may
help.

On proprietary systems you have no choice but keep up with limited
options, on free software you do have choices, and normally you end up
in creating your really personal setup that is so personal to surely
not fit much others, that’s the reason we do not have “slicing
suggestions” out-of-the-box.

Even your personal taxonomy and automation is normally so personal that
does not work for others, for instance typical “modern” guys have tons
of binary files divided per application (like raster image, vector
images, videos, music, …) in a vertically simple (not much nested
dirs), horizontally big (many files/dir in a single subdir), classic
unix guys tend to have tons of text spread across complex (but easy to
traverse for them) vertically big taxonomy, Emacs guys tend to have few
bigger files normally not much categorized since Emacs have a so easy
bookmark system that you tend to ignore fs layout.

As this is not explained in the install instructions, it makes
modifying the current config post-facto difficult if you do not have
another disk around and time for hoop-jumping (I don’t).
Well, if you have up-to-date and tested backups so you feel (and be)
safe, you can easy change things in a live system. For instance you
can create new volume like (zsh/fish syntax)

 # move /home/me to /home/me.old
 mv /home/me{,.old}

 # create&mount new volumes and mount in the correct place
 for vol in rpool/home{,/var,/docs,/prjs,/dotfiles}
 do
   zfs create $vol
   zfs set mountpoint=legacy $vol
   zfs mount -t zfs $vol /home/me${vol/rpool\/home}
   echo "fileSystems.\"/home/me${vol/rpool\/home}\" = \n   {  device =
 \"$vol\";\n      fsType = \"zfs\";\n   };" >> /tmp/addToHwConfig.nix
 done

 chown -R me:mygrp /home/me
 mv /home/me.old/* /home/me 
 nixos-rebuild switch

than you have your original data sliced on different volumes, without
locally copy them so without occupy too much disk space. It’s only a
matter of wait the local move on disk… Now it’s only a matter to
choose how to symlink/bind them, for instance with homeManager or
even via bind-mount that in hwconfig.nix appear as

fileSystems."/mount/point" =
    { device = "rpool/vol";
      fsType = "none";
      options = [ "bind" ];
    };

and you are done, all locally, all without secondary disk :slight_smile:

Therefore, I will not be adventuring around boot loaders again any
time soon.
Anyway do not be afraid, even bootloader change happen functionally

Very happy to have a functional system which is so fast and neat!
You have more, you can reproduce your exact system simply copying
your .nix files and install so in case of system disk failure you
can be up in a very little time and nearly zero manual work without
missing things or doing something wrong. You may also provision many
systems in lan with NixOps, like have a builtin Saltstack or Ansible
at system level, you may upgrade from a release to another substantially
issue-less (generally you have a least to adjust few pkgs names in
your config) since if anything goes wrong you still have ancient
version up and running, selectable at boot like OpenSolaris/IllumOS BE.

I also try to gravitate toward free software but I am very time
constrained and sometimes there are no viable alternatives,
particularly when issues like community, reputation, geographic
replication, etc. are at stake and you have limited time and resources
to devote.
IME it’s mostly a matter of knowledge, if you can “study” enough you
can go FOSS way full-stack in many many many cases, right now we have
even 3D parametric CAD good enough to compete with SolidWorks&c, not
a Catia/Nx/Creo level but enough for a small mechanic firm, we have
enough free FEM/FEA/EDA FOSS for not-so-complicated but not even too
simple project, for instance Purism OEM design their phone motherboard
with free software.

Sometime it’s not only a matter of knowledge since for certain jobs
FOSS support is limited and not so easy to use but in the long run
if you have knowledge you gain far more than any proprietary product
mostly because you have no tie, no imposed limitation.

The real and abruptly serious problem we have is that knowledge is
more and more “private”, we have less and less functional universities,
no more public research, even no more “open devices” to work with and
that’s a thing no less dangerous than a big atomic weapon. Fortunately
something start to move, small open OEMs start to appear, more people
who feel the need of FOSS start to “enter the game” with important
results, so I hope for at least a feature with a jailed mass-market
but a small free niche that can stand on it’s own without being
destroyed by not-really-free market.

Technology help this a bit,think about the possibility in the near
future of 3D printers good enough to print metal (low quality steel)
not only plastic, this is far far away from ancient universities and
big companies open research labs but it’s enough for a new start
especially since managerial driven companies keep going down in quality
and are less and less sustainable to a point that most of our industry
will crush in few decades at actual trend…

Freedom is power, only it does not pay immediately, it’s not easy to
get at first and must be defended continuously, however if you can
face the challenge you’ll be re-paid really well.

Sorry for the long post and poor English.

– Ingmar

1 Like