I usually use flake-based remote OS deployment and it is a rather common occurrence that even with small changes (such as adding or removing a single package) the build takes very long time (minutes). Most often the longest build steps are building man-cache and building steam-fhs-run, with the former being much more prevalent.
Note that I mostly use deploy-rs but using nixos-rebuild build --flake behaves equally in this respect.
Interestingly enough, these build steps look rather out of context for some changes which have nothing to do with Steam (or FHS) and donāt contain man pages.
Honestly, I donāt even know where to start with debugging so any tips in this direction are welcome.
Do you have fish? By default it enables documentation.man.generateCaches (because some completion assumes a working man -k), the problem is that it is unbearably slow and has to be rebuilt on every change to environment.systemPackages and the like.
First you need to figure out what exactly itās waiting on because steam-fhs-run is a trvial builder that basically just writes a pre-determined file. I doubt itās taking any longer than a few milliseconds.
First figure out which exact derivations are taking such a long time to build and what they do internally. Nix should be printing their respective drv paths and you can introspect those using nix derivation show.
(Re-)Build those derivations in isolation in order to confirm they actually take a long time to build rather than finishing instantly and something else in the build graph actually taking a long time without printing anything. Speaking of which, to figure out whether itās the second case, does it print anything after the long wait time?
The next thing Iād do is monitor the system resource usage during the abnormally long wait time to figure out which system resource is the botteleneck: CPU, disk IO or network IO.
Thatās kind of my fault as I added this feature. Personally I would have left it off, but IIRC another fish maintainer insisted on having it by default.
Just let you know, that change noticeably reduced my quality of life (and Iām pretty sure for many others) and should be been either reverted or well documented at fish - NixOS Wiki
I donāt know whatās going on really but I enebaled man pages docs because I want to have them and the build times are indeed completely crazy. Considering how computers fast it might as well be infinity. Does anyone have any idea what is going on there? I canāt wrap my head around why would this take so long to build.
If you donāt care about man completion in fish or apropos you can disable the caches. I have no idea why itās so slow, probably no one ever paid much attention to it because on other distribution you can just do that with a cron job in background.
Iām using zsh so I donāt care about fish. It would be nice to have it for apropos but for now I have this disabled which completely fixes the unacceptable performance during rebuilds.
Thanks for the links that answers my question - itās doesnāt seem NixOS specific indeed. There is something very wrong but itās not on our side.
On more standard distros, there is a cronjob or systemd timer that runs a script to update the cache, based on what man pages are currently installed. I guess that this could also be replicated on NixOS. The script should probably be generated, though, to search for all the packages in environment.systemPackages, etc.
). Iāll try disabling the cache generation, thanks!