Hi there – first-time poster, but have been using NixOS exclusively for ~8 months.
I’ve recently moved my flake from unstable to 24.11, and I’m having a hard time getting it to build (via nixos-rebuild) on an existing system, or install (via nixos-install) on a new one.
There’s no evaluation errors, but the build will run, maxing out CPU/memory, and the system becomes unstable. If running in tmux, tmux will exit. If on a normal emulator shell, the emulator will exit. This does not stop the build processes though, which will continue until I manage to kill enough, or reboot the system. Keyboard and mouse can become unresponsive also, forcing a physical reboot.
My desktop (Ryzen 3600X, 24GB RAM) is currently attempting to build – it’s been running for two hours now, and I cannot unlock it with the keyboard and mouse due to being unresponsive. I’ve been able to ssh in and this is what btop looks like:
I’m guessing that tmux/terminal exits are caused by OOM, but otherwise I’m unsure how to debug or diagnose, or – if it is just a large build, constrain the build processes so that they don’t make the system unusable in the process or kill their parent shell.
In the list of processes, I have just noticed a lot of the cc1plus instances are related to Telegram, so I shall try removing that and report back.
Well yes, that’s how running out of memory typically looks like. Though I’m a bit surprised it’s killing tmux before the nix build; that shouldn’t happen.
You should also add some swap. Either physical XOR zram swap.
I have zramSwap.enable = true and tmp.useTmpfs = true (with tmp.tmpfsSize = 30%). Perhaps I’m asking too much of 24GB RAM during larger builds?
I killed the build on my desktop, commented out telegram-desktop and re-ran, and noticed ~35 threads of .java-wrapped from a store path for temurin-bin (I guess, a dependency for something). I had the build running over ssh and the last thing in the terminal before the ssh connection got killed by OOM was electron-unwrapped-33.3.0 (buildPhase): [2199/41550] AR obj/third_party/icu/libicuuc.a. This time the OOM killed the window manager, but building is still ongoing.
Thanks, both, for the tips – I’ll take tmp out of tmpfs (though I have nix building out of this directory as this was previously cancelling builds due to running out of space. It’s currently only got about 4KB out of 7GB used) and change zram swap for physical swap space, and also check out @waffle8946 's link for tuning the build jobs
Edit: Related: this post, will try the recommendations there, also.
So it turned out that electron-unwapped (both 22.something and 23.3.0) were the causes of the OOM. I found one package, bruno, that was the dependent on electron.
In the end I ended up adding nix.settings.max-jobs = 1 to my flake and sitting through the ensuing long build.