Reduce "nix-env -u" memory usage?

Help
1

Currently, “nix-env -u” uses huge amounts of memory: about 11 GB just for the “evaluating file”-step!
So, “nix-env -u” is completely unsusable on all machines with <16GB of RAM! (or: less then 11GB free RAM!).

Before submitting a bug report: Is there any way to reduce the “nix-env -u” memory usage to less then 4 GB with nixos-23.11?

Details:

  • According to top, the “evaluating file”-phase of nix-env -u uses 10.9GB of RAM (!!).
  • Virtual Server with 4GB RAM (actually 2GB RAM + 2GB swap, with 3.5GB free at the beginning):
    • With nixos-23.05, nix-env -u worked.
    • With nixos-23.11, it fails:
# nix-channel --list
nixos https://channels.nixos.org/nixos-23.11
# nix-env -q

# time nix-env -u -v
evaluating file ...
...
evaluating file ...
Killed
1 Like
2

Looks like there’s previous discussion of this here: nix-env uses a ton of memory · Issue #38635 · NixOS/nixpkgs · GitHub

3

nix-env -u in every variant, evaluates the entirety of nixpkgs trying to find what needs to get updated. That requires, as you say, large amounts of memory. It’s time to share this classic link: https://stop-using-nix-env.privatevoid.net/ (which links to what @srd424 shared); the article suggests using nix profile instead for imperative package management.

1 Like
4

So, nix-env does use a lot of memory, but I’m unable to reproduce an 11G sized problem. When I measure the resident memory usage with \time -f '%M' nix-env -u (coreutils time, not shell builtin time), I get a max residency of 2.5G Still a lot, but nowhere near 11G.

5

I‘d actually go as far as saying that imperative environment management is an unfinished feature and I don’t recommend going down that path, unless you really don’t care about customisation or control – and I assume that’s unlikely given the average quality of open source software.

1 Like
6

Yes, I should have shared that I don’t use nix profile and really only use nix-env to list my generations and and delete them when I need to. There’s probably a nix-command to do that but force of habit :man_shrugging:

7

Unfortunately, “nix profile” is marked as “experimental” and disabled by default, which sounds to me that the nix-authors do not recommend it…

8

Experimental features don’t say anything about recommendations, they’re about stability guarantees. Anything that’s marked stable is not supposed to change backwards-incompatibly. Experimental features allow for essentially arbitrary changes so they can get into a state where long-term support and maintenance is tenable. See the Nix manual for reference:

https://nix.dev/manual/nix/2.19/contributing/experimental-features

Recommendations are, as far as I can tell, more of a matter of personal opinion.