Greetings everyone! I’ve ran into a problem which I hope it’s solvable without having to reinstall the system: My nix store has grown tremendously large to the point I can’t rebuild, upgrade or update the system anymore. I’ve followed the wiki pages on how to optimise the store, run GC, clear symlinks and older generations, etc, but none of these are able to free any space, even a single MB, let alone GB. I haven’t installed many packages, so I can’t wrap my head about what even is taking all that space (/nix is taking 31GB out of the 65GB overall, and if I try to run rebuild switch it’ll just fill the rest of the 65GB and fail). This is NixOS 24.05, on 6.8.11_xanmod1. If anyone here has seen this issue before, and knows what’s up, please, help me out! Thank you all in advance for taking your time to read my post, and for your help.
65 gigs is ridiculously tiny, you’ll want at least a couple hundred gigs.
Boot a live image and resize first.
I’m not sure I agree that 65G is tiny. My system closure is only 14.8G, and I’d probably call that sort of medium sized for a nixos desktop.
@TheConundrumer Have you made sure to clear old generations from your user’s profile, from the root user’s profile, and from the /nix/var/nix/profiles/system profile? Or have you checked nix-store --query --print-roots for stray roots?
64 gigs should be enough. Are you using flakes and copying extra stuff into store like in this thread?
Everything’s relative.
I just did a du -sh /nix and it turns out I’m using 93GB. But in there is a full 30d of rollback history.
Yeah, everything has been cleaned, I was left with only the current generation
Nope, I haven’t used any flakes on my system (yet). I did try many different kernels and DEs in the past though, before settling with plasma and xanmod. I was considering the possibility of all that space being taken by older kernels and left overs, especially because at some point I did run custom kernels, with custom flags. But it wouldn’t make sense these still being around, especially after running GC so many times, would it?
Your runtime closure has little bearing on the size of the nix store you’ll need.
First off, you have to account for multiple generations; even with store optimisation on, after (say) a staging merge you may see very little overlap between generations, meriting a significantly larger store.
I’m speaking from experience btw, with an even smaller closure and store - I ran a system with a <7 gig closure and 40 gig /nix partition. Deleting all previous (non-current) generations and trying to switch to a new generation after a staging merge was simply insufficient.
Secondly, your buildtime requirements may be larger than you think; just because your system closure is a certain size, doesn’t preclude needing to fetch even larger sources (and if using e.g. fetchzip, the extracted sources would be persisted in the store).
And if you ever even want to think about contributing to nixpkgs, forget about having any space for that.
65 gigs is simply not enough.
Anyway other things to try:
- Ensure you’re deleting the previous generations as root and then gc. Or you can do it in one step with
sudo nix-collect-garbage -d(replace-dwith--delete-older-than <whatever>as you please, if you really need some of the older generations).- If you’re using something e.g. home-manager that manages user profiles, ensure to repeat the above as non-root - user and system profiles are independent.
- If you’re building stuff locally, i.e. some packages aren’t cached by hydra, make sure your
/tmpis on a disk with sufficient space (/tmpis not on a tmpfs on nixos by default, nor is it wiped by default; you need to clear that out if it’s taking up space). - Check for extra gcroots that you may be able to delete manually, especially if you did some manual builds.
- The last-resort option would be to boot a live image, wipe the entire store partition (don’t wipe anything else!), and “reinstall”. This would not delete your data, as runtime data is not stored in the nix store. Ensure to mount everything, including the store, relative to
/mnt(or pass--root <path>tonixos-installif you want to use a different root, and mount relative to that). However, this will prevent you from booting older (including current) generations.
I might have to go with the last-resort option. The sudo nix-collect-garbage-d says it has deleted 0 files and freed 0MB, I also only have one gen as I’ve already deleted all the previous ones (current is 617). Is there a way to find out what exactly is filling it up like that? The nix store wasn’t that big initially, with a size of only ~20GB back in 23.11. But it kept growing with each rebuild, since the update to 24.05, which probably means there’s of left overs, but the GC always says it has freed 0MB even running as sudo. I have the same packages as I had back in 23.11, barely changed anything in the system since the update: only kernels, DEs, nerdfonts and swiching Nvidia to AMD. I did build some kernels locally, many times.
I use nix-tree (3rd party tool) to investigate what the biggest parts of the system closure are. You can pass --derivation flag to check buildtime closure, or omit it for runtime closure.
Also since you mentioned nerdfonts: how are you installing it? Are you overriding it? And which DEs do you have installed?
I have a raspberry pi cm4 with 8g eMMC, runtime closure is barely over 4.5g. I think it went up from 3.7 to 4.5 after upgrade to 24.05, so there might be some merit to hypothesis that 24.05 needs more space than 23.11 for identical config.
I also used nix-tree combined with why-depends at some point to pare down the store space usage. That’s on top of disabling as much state as possible.
If anyone finds this post later - don’t run this setup, get some proper storage. eMMC is super slow and I need to disable certain packages, build the system, collect garbage, enable the pqckages back and redeploy the machine on ~bikweekly basis.
Use the aforementioned nix-store --query --print-roots to find out what paths aren’t being garbage collected.
How large is your closure? (nix path-info -Sh /run/current-system)
Sorry for taking too long to reply. nix path-info -Sh /run/current-system returned 29.7GB;
nix-store --query --print-roots returns an error saying --print-roots is an unknown flag.
I just declared nerdfonts as an user package directly on my config file. Also, I have been through XFCE, Hypr, Hyprland and Enlightenment before finally settling with Plasma 5.27.11 on X11 (removed all previous DEs from my configuration.nix too).
Well there you have it: One instance of your system takes ~30GB. No amount of garbage collection will make your nix store any smaller.
As @ElvishJerricco mentioned, it should be half of that for a typical desktop system. Even my quite bloated gaming system closure is “only” 22GB and that has like 3 different JDKs and a typical desktop closure for 32bit in it in addition to a decently sized desktop closure.
I’d recommend you check what’s bloating your closure so much using nix-tree.
Ah, it’s nix-store --gc --print-roots. Though as I said, investigating this isn’t necessary as your nix store only consists of ~1 closure, it’s just the closure being huge.
That’s part of the mistake. nerdfonts is 5 gigs, you definitely want to override it and only select the fonts you use.
Got it, just removed it from the config file, I’ll admit that out of all nerd fonts I only used like 3 of them. I just thought about shrinking my home partition, and so I did by 100GB. How can I extend the root partition so it takes that extra 100GB? Or perhaps move the nix store itself over there? I feel like this might be an easier solution…
That depends entirely on your storage setup and would be better asked/discussed in a generic Linux forum or answered by a little bit of research on your end.
If you want to know what is using all the space,
du -hac /nix/store/ --max-depth 1 | sort -h
will show you exactly what’s in the store and what the largest packages are. Or you can use something like
nix run 'nixpkgs#gdu' /nix/store
These don’t know about roots or system generations, but they will show without a doubt what is eating up all the space.