Looking specifically at nixos.closures.ec2.x86_64-linux
it looks like documentation generation is quite a heavy hitter and spends a bunch of time doing source filtering:
GC_INITIAL_HEAP_SIZE=8g nix-instantiate nixos/release-combined.nix -A nixos.closures.ec2.x86_64-linux --eval-profiler flamegraph --eval-profiler-frequency 10000
I wouldn’t be surprised that the overhead mostly comes from string comparisons for the type string (a.k.a. “directory”, “symlink” e.t.c.). Those symbols are stored in the SymbolTable
, but the string comparison could no doubt be optimized (e.g. when allocating a string we could first check if it’s available in the SymbolTable
) and then just skip the allocation altogether (as in Optimize symbol table by NaN-git · Pull Request #13258 · NixOS/nix · GitHub). With that string comparison could just become a simple pointer equality check for those strings.
As for the stdenv eval being slow we have by-name-overlay
to blame partly:
Since nix doesn’t provide an effecient list of attrs merging builtins nixpkgs has to resort to rolling our own binaryMerge
. We can do much better in the evaluator (I’ve build upon Robert’s work from Prototype: accumulate attrset updates, perform k-way merge by roberth · Pull Request #11290 · NixOS/nix · GitHub here: Commits · xokdvium/nix · GitHub and was able to shave off some time spent in the by-name-overlay merging). By providing a more efficient builtin like mergeAttrsList
we could somewhat mitigate the by-name
performance implications. (haven’t yet pushed the builtin PoC to the branch). On the other hand, maybe the need for eager attrset merging that has to be done by the overlay is far from optimal. There were some prior art about making attrs lazy by attribute names: Lazy attribute names · Issue #4090 · NixOS/nix · GitHub