I use shell.nix + direnv extensively to manage project dependencies, and my store looks like this, I tried nix-collect-garbage -d but it doesn’t do much, saving maybe 1GB or so.
So I’m looking for ways to optimize this more if possible…
The first place to start is nix-store --gc --print-roots
That will show you all the places that are currently holding things alive, which for your case will likely include a lot of project directories with .direnv caches.
If a project is idle, you can just remove that directory and let the store references be gc’d. In all probability, when you next pick up the project, your first step will be to update to current inputs anyway.
1 Like
This works, but there is no way to some form of deduplication for example?
Hi. There is a built-in mechanics for file-level deduplication for the nix store. See Nix Cookbook - NixOS Wiki
1 Like
Thanks will have a look at this
Personally, I use zfs deduplication on the nix store dataset, but that’s obviously dependent on system config and capability. It just means I’ve never considered using the nix hard-linking mechanism instead.
However, while it will help (with either mechanism) for versions that are “close” to each other, it’s not so useful where you have lots of projects with cached dev shell deps that you visit sporadically: over time these will include a wider version-spread of tools that are sufficiently different that they don’t dedup as effectively.
So I think the best case is “a bit of both”.
1 Like
Ok so I have tried two approaches from the ones mentioned in the thread and here are my findings:
-
nix-store --gc --print-rootsand removing a bunch of.direnvcaches: works but doesn’t really save a lot of space (I even experimented with removing all.direnvcaches) - Setting
auto-optimise-store = trueis not an option because it’s buggy on darwin "error: cannot link '/nix/store/.tmp-link' to '/nix/store/.links/...': File exists" · Issue #7273 · NixOS/nix · GitHub
In fact my store grew in size and now became 310GBs so not sure where I go from here…
@uep I can’t do this on this machine.
Did you try to run sudo nix store optimise to deduplicate files in the store? It does work even with the “cannot link” issue you are referring to.
Just as sanity double check, did you gc after removing the gc-roots?
Which gc roots are active?
Possibly nix-du is worth looking at to visualize the roots dependencies.
@Mayeu I’m running this tonight overnight to see if it will help
@Shawn8901 yes I did. Will have a look at nix-du, thanks
1 Like
I ran nix store optimize overnight, which took ~10hrs, and removed 107GBs so that was useful
1 Like