kkflt
March 3, 2025, 4:42pm
1
Is there a way to limit cpu usage / load when using nixos-rebuld switch
? I’m not using the binary cache, and I keep getting OOM’ed ( the load average reaches over 70 on my 8 core cpu )
phaer
March 3, 2025, 5:15pm
2
Your title is about CPU, but getting OOM’ed suggests it might (also) be about memory usage? This older thread could be helpful here Nix build ate my RAM 😭
kkflt
March 3, 2025, 5:22pm
3
Yes, its running outbof memory because it tries to do too much. I would like to limit the load to 8
ookhoi
March 3, 2025, 5:24pm
4
nix = {
settings = {
cores = 8;
};
};
Lun
March 3, 2025, 5:34pm
5
For now cores and max-jobs are the options you can tune.
There’s a stalled attempt to allow configuring a load limit separately.
NixOS:master
← emilazy:push-tyzwwmvlwvwo
opened 02:27PM - 20 Jul 24 UTC
# Motivation
This is a rework of https://github.com/NixOS/nix/pull/6855. It i… sn’t a perfect solution (e.g. if you're building a bunch of Rust stuff they’ll still use as many cores as they want), and it’d be great to have an implementation of the new and improved GNU Make job server protocol at some point, but it seems like there’s widespread desire for this to come back in some form as it provides a much better UX for workstation users doing large builds, especially when you don’t have copious amounts of spare RAM. I have a draft Nixpkgs stdenv change to use this variable that I will post and link here.
I would like to backport this down to 2.18 so that NixOS and nix-darwin users get a good experience out of the box. I can handle the manual backport work for any versions the bot fails on.
# Context
Some notes and questions:
1. I kept the default at `getDefaultCores()` rather than using the `cores` setting. That seems like a reasonable middle ground: system load is a bit of a laggy, imprecise measure, so using `cores` directly would likely make CPU go to waste by default, but most workstation users probably want to keep total CPU utilization below 100% so they can still use their machine. This way, we don’t regress on the concerns laid out in https://github.com/NixOS/nixpkgs/pull/174473 and https://github.com/NixOS/nixpkgs/pull/192447. It’d also be kind of a fuss code‐wise to make the default depend on another setting like that. I’m open to bikeshedding on this, though; `none` and the value of `cores` also seem like plausible defaults, although I think the former would be optimizing too much for Hydra over the UX for people with normal machines that aren’t exclusively used for build farms. Given that the default for `cores` is to use all the cores on the machine, and that limiting `cores` doesn’t limit total system load at all right now, I think this is probably fine.
2. I’m a bit unsure about how `getDefaultCores()` interacts with remote builders, and the difference between having `cores` unset and setting `cores = 0`. If you have remote builders and the default `cores`, does each remote builder limit to the number of cores on the machine doing the evaluation? Surely not, right? The Nixpkgs stdenv [explicitly handles `cores = 0`](https://github.com/NixOS/nixpkgs/blob/f22436e2b41d758510d8132387b26d91937a80e4/pkgs/stdenv/generic/setup.sh#L856-L865), normalizing it to the result of `$(nproc)`. Is this just a backwards‐compatibility hack and there’s no reason for `cores = 0` these days? I want to understand if there’s any reason for both `cores = ‹default›` and `cores = 0` to exist, so that I can understand if it makes sense to mirror this behaviour with `load-limit = 0` as my current changes do.
3. GNU Make and Ninja have inconsistent interpretation of `0` and `-1`; with GNU Make `-l0` means something you’d never want and `-l-1` disables the limit, and with Ninja `-l0` disables the limit and I think `-l-1` might be invalid. I decided that it would be better to have an explicit non‐integer value to disable the load limit entirely, and to make `load-limit = 0` the same as `cores = 0`, but obviously (3) is a load‐bearing consideration here.
4. Relatedly, is `load-limit = none` okay? Nothing uses that syntax currently. It seemed less obscure than using the empty string.
5. I don’t understand what the `src/libstore/daemon.cc` thing is doing. I copied it from https://github.com/NixOS/nix/pull/8105. Please tell me what it does and if it’s good or bad!
Closes: #7091
Closes: #6855
Closes: #8105
cc @fricklerhandwerk @centromere @aviallon @reckenrode @ElvishJerricco
# Priorities and Process
Add :+1: to [pull requests you find important](https://github.com/NixOS/nix/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc).
The Nix maintainer team uses a [GitHub project board](https://github.com/orgs/NixOS/projects/19) to [schedule and track reviews](https://github.com/NixOS/nix/tree/master/maintainers#project-board-protocol).
3 Likes
kkflt
March 3, 2025, 9:23pm
6
So I found you can do nixos-rebuild switch --options cores 1
. I haven’t tried the rest of solutions but I’m sure they also work. Thanks
flake version:
nixos-rebuild switch --flake .#nixos --option cores 3 --option max-jobs 6 --option max-memory-size 8G
1 Like
If you have fast ssd storage to spare, I suggest trying out the swapspace module.
this doesn’t work: max-memory-size 8G