Excessive memory use during eval of a particular flake

This is happening with a particular flake I’m using and I tried getting help for this at Excessive memory use during eval when upgrading to new esp32 version · Issue #10 · bouk/arduino-nix · GitHub , but maybe someone here can offer some thoughts:

Until recently I’ve been using arduinoPackages.platforms.esp32.esp32."3.1.3" from the GitHub - bouk/arduino-nix: Manage your arduino-cli with Nix flake and this has worked perfectly. However, recently I have tried to update to version 3.3.0 and my poor CI machine suddenly started getting OOM problems.

I’ve put a simple reproducer here: GitHub - SFrijters/nix-arduino-eval-mem-usage . It uses “3.2.0” because that’s the first version where it exhibits the problem.

For me, nix build .#esp-3_1 works fine, with maybe a couple of hundreds of MB of memory usage, but when running nix build .#esp-3_2 it quickly balloons into multiple GB before being killed on my resource-limited CI machine. On my regular desktop the process eventually completes.

I’ve tried upgrading my nix version to 2.30 because of its claimed memory savings, but this hasn’t solved the issue.

If someone can reproduce this issue that would be great. I don’t see what the big difference is between the two versions: the json index files both refer to similar stuff, just with different versions, so I don’t understand why this would make such a difference to the nix eval process.

[ci-machine]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.12.43, NixOS, 25.11 (Xantusia), 25.11.20250825.3b9f00d`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.30.2`
 - channels(root): `""`
 - nixpkgs: `/nix/store/r33xw4w5nmpmfac4g47h1lmrcl4qlz7k-source`
 - ```

It looks like both the memory usage and file system inputs have increased significantly:

[ci-machine]# nix run .#time -- ".#esp-3_1"
/nix/store/3z5dwq8fsr1hc5628xi0r1gp6crxr9nm-arduino-cli-wrapped	Command being timed: "nix eval --trace-function-calls --raw .#esp-3_1"
	User time (seconds): 0.75
	System time (seconds): 0.15
	Percent of CPU this job got: 97%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.93
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 203012
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 2
	Minor (reclaiming a frame) page faults: 45603
	Voluntary context switches: 71
	Involuntary context switches: 12
	Swaps: 0
	File system inputs: 3808
	File system outputs: 288
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

[ci-machine]# nix run .#time -- ".#esp-3_2"
Command terminated by signal 9
	Command being timed: "nix eval --trace-function-calls --raw .#esp-3_2"
	User time (seconds): 1.93
	System time (seconds): 8.19
	Percent of CPU this job got: 52%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:19.22
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 1553796
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 21616
	Minor (reclaiming a frame) page faults: 711489
	Voluntary context switches: 1343
	Involuntary context switches: 3017
	Swaps: 0
	File system inputs: 172976
	File system outputs: 280
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

Any help would be welcome!

Are you sure it’s during eval rather than during build? I don’t see a memory spike during eval.
Maybe try running nix eval instead of nix build to confirm what’s happening here.

The run command is just a wrapper around a nix eval invocation with a particular nix, but if I just run the raw commands on my CI machine the OOM killer strikes again / still for me:

[root@<redacted>:~/nix-arduino-eval-mem-usage]# nix eval .#esp-3_1
«derivation /nix/store/c83h94v80m4fbmkp3naykx5x0l1q1yj5-arduino-cli-wrapped.drv»
[root@<redacted>:~/nix-arduino-eval-mem-usage]# nix eval .#esp-3_2
[566.2 MiB DL]Killed                     nix eval .#esp-3_2
[root@<redacted>:~/nix-arduino-eval-mem-usage]# command time -v nix eval .#esp-3_1
«derivation /nix/store/c83h94v80m4fbmkp3naykx5x0l1q1yj5-arduino-cli-wrapped.drv»
	Command being timed: "nix eval .#esp-3_1"
	User time (seconds): 1.03
	System time (seconds): 0.42
	Percent of CPU this job got: 63%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.28
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 239728
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 245
	Minor (reclaiming a frame) page faults: 54189
	Voluntary context switches: 3574
	Involuntary context switches: 340
	Swaps: 0
	File system inputs: 180728
	File system outputs: 240
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
[root@<redacted>:~/nix-arduino-eval-mem-usage]# command time -v nix eval .#esp-3_2
[566.2 MiB DL]Command terminated by signal 9
	Command being timed: "nix eval .#esp-3_2"
	User time (seconds): 1.62
	System time (seconds): 8.86
	Percent of CPU this job got: 71%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:14.65
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 1636632
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 20320
	Minor (reclaiming a frame) page faults: 731937
	Voluntary context switches: 1039
	Involuntary context switches: 3503
	Swaps: 0
	File system inputs: 177920
	File system outputs: 224
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
[root@<redacted>:~/nix-arduino-eval-mem-usage]# nix --version
nix (Nix) 2.28.5

One thing’s for certain, it’s not using up the memory during eval:

NIX_SHOW_STATS=1 NIX_SHOW_STATS_PATH=stats.json nix eval "github:SFrijters/nix-arduino-eval-mem-usage?ref=95bd5624de530dd871f8bc4280ba7ad0fbd2bc46#esp-3_2" && jq ".gc.totalBytes" stats.json
«derivation /nix/store/1dd7x3a3387ks8y8pa2dbg53bi0a429y-arduino-cli-wrapped.drv»
92922464
NIX_SHOW_STATS=1 NIX_SHOW_STATS_PATH=stats.json nix eval "github:SFrijters/nix-arduino-eval-mem-usage?ref=95bd5624de530dd871f8bc4280ba7ad0fbd2bc46#esp-3_1" && jq ".gc.totalBytes" stats.json
«derivation /nix/store/c83h94v80m4fbmkp3naykx5x0l1q1yj5-arduino-cli-wrapped.drv»
92922464
3 Likes