Activation scripts takes a long time

Since a few days ago the activation scripts when booting takes a long time, last boot it was 8+ minutes and latest nixos-rebuild switch was 16+ minutes. I found this post on the forums: https://discourse.nixos.org/t/running-activation-script-takes-several-minutes and it seems to be the same perl script (update-users-groups.pl) that takes a very long time to complete. The file suggested in that thread (/var/lib/nixos/uid-map) looks normal for me and removing it does nothing.

systemd-analyze outputs:

Startup finished in 11.178s (firmware) + 2.801s (loader) + 8min 1.940s (kernel) + 6.591s (userspace) = 8min 22.511s 
graphical.target reached after 6.591s in userspace

Any suggestions on how to troubleshoot?

My configuration can be found at http://github.com/simonkampe/nixos , it is the host named feynmann.

Update, now the activation script errors out instead:

nixos on î‚  main [!] 
❯ sudo nixos-rebuild switch --flake ".#"
[sudo] password for simon: 
warning: Git tree '/home/simon/Workspace/nixos' is dirty
warning: Using saved setting for 'extra-experimental-features = nix-command flakes' from ~/.local/share/nix/trusted-settings.json.
building the system configuration...
warning: Git tree '/home/simon/Workspace/nixos' is dirty
warning: Using saved setting for 'extra-experimental-features = nix-command flakes' from ~/.local/share/nix/trusted-settings.json.
activating the configuration...
panic: pp_match start/end pointers, i=1, start=0, end=3758096461, s=7fa00a74f010, strend=7fa0ea74f05d, len=18446744073172680781 at /nix/store/c9aks0l4kz0alyq4g64xjx8mk7ps3dk7-perl-5.34.1-env/lib/perl5/site_perl/5.34.1/File/Slurp.pm line 133.
Activation script snippet 'users' failed (255)
setting up /etc...
reloading user units for simon...
setting up tmpfiles
warning: error(s) occurred while switching to the new configuration

That’s 3GB of matching! Looks like some script goes off to read an absolutely massive file that probably isn’t supposed to be parsed?

For future reference, I hunted the file down using:

du -cks * | sort -rn | head

The file was “/var/lib/nixos/auto-subuid-map” and it was filled with garbage. I deleted it and did nixos-rebuild switch and now it seems to be back to normal.

2 Likes