"nix-env -i hello" vs "nix-env -f '<nixpkgs>' -iA hello"

For some reason the command, “nix-env -i hello” takes a lot of memory (A LOT). I’ve used nix on Windows WSL and on Oracle Virtual Box. On WSL the command does not output anything for a whole 5 minutes and then starts downloading. On the Oracle VM since the ram given is less it times out.

On the other hand “nix-env -f ‘<nixpkgs>’ -iA hello” works like a charm. :slight_smile:

Why does this happen? What seems to be the problem?

PS: I started out with Nix recently. I hope this is the right place to ask such questions.

4 Likes

nix-env -i hello searches through all of nixpkgs to find the derivation with the name attribute that matches hello (can even be a regex).
nix-env -iA hello searches the top-level attributes of nixpkgs to find the one at hello, it does not look inside the derivations. That’s why it’s faster.

For example the top-level attribute for GNU hello is defined in pkgs/top-level/all-packages.nix at hello = callPackage ../applications/misc/hello { };.

Its name attribute is defined in pkgs/applications/misc/hello/default.nix at

{ stdenv, fetchurl }:

stdenv.mkDerivation rec {
  name = "hello-${version}";   # <-------
  version = "2.10";

P.S. have a look at the manual nix-env(1) :wink:

2 Likes

Hi, Thanks for the reply! I’m actually concerned. As mentioned earlier nix-env -i hello takes a lot of memory and time. This should not be normal right? Is this a problem with WSL? I have a decent system with 8GB of RAM. But still, if the basic installation method is this heavy how can I use it for everyday tasks?

Ideally, I should only be working with nix-env --install

Even on my NixOS system nix-env -i hello takes 1.2GB of RAM and 12 seconds to execute. Yes, that is a lot but also our repository is huge. We have 65445 name attributes in 14337 files which all need to be parsed. (I’m not sure what kind of optimizations nix-env does or how it looks for the names).

$ find pkgs -name "*.nix" | xargs cat | grep 'name = "' | wc -l
65445
$ find pkgs -name "*.nix" | wc -l
14337

Usually you don’t need the plain -i if you don’t care about searching for the name with a regex. Usually name and attribute path are the same or very similar.

How many channels do you have configured?

On my macOS machine, with a single nixpkgs-unstable channel, nix-env -qa hello takes approximately 30 seconds and a maximum RSS of 1.1GiB.

On my Linode running NixOS with 2GB RAM and 2 channels configured (nixos-19.03 and nixos-unstable), nix-env -qa hello runs out of memory after 13 seconds and 1.7GiB maximum RSS.

I’m not actually sure why it requires such a high memory ceiling. I expect it’s holding onto the full package attribute set, forcing all of the thunks in turn. Perhaps we should try something like constructing a list of all the values, and then popping each value off the list in turn before evaluating the derivation, such that once it’s calculated the name there’s nothing pointing at the derivation anymore and thus it can be collected. This would still produce a lot of GC churn, but it should reduce the maximum memory usage.

Although to be honest what we really should do is have hydra precalculate the entire list of name = attribute-path for every package that nix-env -q would search, and then have nix-env just check that. It can fall back to its current behavior for any channel that doesn’t include the list. The downside here is Hydra would have to calculate that list separately for each platform, otherwise packages that bail out with asserts on some platforms would not show up properly.

3 Likes

@lilyball Sorry for the delayed response. Was very busy the past few days.

I have 1 channel configured which is nixpkgs-unstable. As, @JohnAZoidberg mentioned, I understand that the repository is huge and hence it takes a lot of memory. On top of that, I’m running it on WSL which might not be optimal. I’ll run a few experiments and get back to this discussion. I’m really interested to get into Nix, it’s very interesting. And once I get the hang of Nix, I’d like to move to NixOS completely.

Thanks for the help :slight_smile:

1 Like

FWIW I’ve found nix search to work pretty well. You do have to remember to ask it to update its cache every time you update your channel, but other than that minor annoyance, it runs just fine on my NixOS machine. And then once you’ve found your package you use nix-env -iA to install it via attribute path.

3 Likes

https://nixos.wiki/wiki/FAQ/Why_not_use_nix-env_-i_foo%3F

2 Likes

Thank you! This helps a lot. Wouldn’t it make sense to put this in the manual? Honestly the biggest reason I hadn’t started using Nix months ago was the fact that the nix-env -qa firefox took over 1GB of RAM ( not something I had an abundance of at the time ). Having nix search and knowing to nix-env -iA instead of nix-env -i would have made a big difference for me.