Show closure size of garbage collector roots

StillerHarpo · June 14, 2023, 8:17pm

Is there some tool or command to list the summarized closure size on all garbage collector roots?

StillerHarpo · June 14, 2023, 8:36pm

With the help of [1] I could come up with something myself:

for p in /nix/var/nix/gcroots/per-user/florian/*
do
   echo -n $p" ⇒ "
   nix-store -q --requisites $p | sort -uf | xargs du -ch | tail -1
done

[1] nixos - How to get the size of a Nix derivation? - Unix & Linux Stack Exchange

sternenseemann · June 14, 2023, 9:20pm

We can also use the already built in functionality of nix path-info -S to get the closure size of a store path. Additionally we can use GNU find to get the all gcroots (not just for your user). Bonus formatting of the resulting size with GNU numfmt.

find /nix/var/nix/gcroots/ -type l -readable \
  | xargs nix path-info -S \
  | awk '{ sum += $2; }; END { print sum }' \
  | numfmt --to=iec --suffix=B --format="%.2f"

uep · June 14, 2023, 9:51pm

This summation to a single total is counting a lot of stuff multiple times, as there will be quite a bit shared between system derivations, and probably is also hiding the information originally wanted (which gc roots should I delete to try and clean up).

~~Also minor formatting nit, the final closing ` should not be there (it is presumably a stray markdown formatting quote)~~

sternenseemann · June 14, 2023, 11:00pm

Right, we’d need to plug nix-store --requisites and sort -u in between and then use just nix path-info -s to ignore the closure size.

find /nix/var/nix/gcroots/ -type l -readable \
  | xargs nix-store -q --requisites \
  | sort -u \
  | xargs nix path-info -s \
  | awk '{ sum += $2; }; END { print sum }' \
  | numfmt --to=iec --suffix=B --format="%.2f"

StillerHarpo · June 15, 2023, 1:49pm

Thanks @sternenseemann
Inspired from your command I came up with this script, which shows all gc roots and their size (which are most likely overlapping.

for link in $(find /nix/var/nix/gcroots/ -type l -readable)
do
  res="${res}
$(readlink "${link}") $(nix path-info -Sh "${link}")"
done
echo "$res" | sort -h -k3,3

``

symphorien · June 15, 2023, 9:39pm

nix-du outputs a graph that shows the size attached to each gc root and the amount of space shared between them.

picnoir · November 21, 2024, 8:34pm

Just in case anybody stumble upon this from a search engine like I did, here’s a variant of StillerHarpo’s solution:

#!/usr/bin/env bash

# Collecting GC roots paths
declare -a roots
while IFS= read -r -d '' link
do
        roots+=( "$( readlink -f "$link")" )
done < <(find /nix/var/nix/gcroots/ -type l -readable -print0)

# Retreiving GC roots sizes, order by size
tmp="$(mktemp)"
trap 'rm $tmp' EXIT
nix path-info --closure-size --json "${roots[@]}" | jq 'sort_by(.closureSize) | map({ path: .path, size: .closureSize })' > "$tmp"

# Pretty print result
jq -r '.[] | .path + ": " + (.size | tostring)' "$tmp" | while read -r line; do
    path=$(echo "$line" | cut -d: -f1)
    bytes=$(echo "$line" | cut -d: -f2 | tr -d ' ')
    
    # Pretty-print bytes to a more appropriate human format
    readable_size=$(numfmt --to=iec-i --suffix=B --format="%.1f" <<< "$bytes")
    
    echo "$path: $readable_size"
done

uep · November 22, 2024, 1:49am

and here’s a nushell version (edit: tiny update to work with current nu as it evolves)

def gcdu [
    --processes (-p)    # also show roots from running processes
]: nothing -> table {
  let roots = ^nix-store --gc --print-roots 
    | parse "{root} -> {closure}"
    | where root !~ '{censored}|^/proc' or $processes
    | group-by --to-table closure 
    | rename path roots
    | update roots { get root | sort | str join "\n" }
  
  let pathinfo = ^nix path-info --closure-size --json ...($roots | get path) 
    | from json
    | select closureSize narSize path
    | into filesize closureSize narSize
  
  $roots | join $pathinfo path
    | update path { path basename }
    | rename -c { path: "store path" }
    | sort-by -r closureSize
}

in addition to the shell/syntax differences, this version uses a different method to find the roots in a way that shows the thing(s) you want to delete in order to let the closure get gc’d, and groups them together.

sample output, in md format (slightly modified because discourse’s md tables don’t do inner newlines)

❯ gcdu | take 10 | to md -p

store path	roots	closureSize	narSize
qvz2lfr2513g26m41h8jrfy9p9vwz62i-nixos-system-oenone-24.11.20241111.dc460ec	/nix/var/nix/profiles/system-557-link	31.9 GiB	157.5 KiB
lbm6m1q8jg9lc02fwjs79x6jx64a1k5z-nixos-system-oenone-24.11.20241109.76612b1	/nix/var/nix/profiles/system-556-link	31.9 GiB	157.5 KiB
94dh5jxqfbknbpyvx3dmkswvf97n9pgf-nixos-system-oenone-24.11.20241109.76612b1	/nix/var/nix/profiles/system-555-link	31.9 GiB	157.5 KiB
611ma22g3azfzify0yc2sxyshwr4p4kv-nixos-system-oenone-25.05.20241115.5e4fbfb	/nix/var/nix/profiles/system-559-link /run/booted-system /run/current-system	29.4 GiB	157.5 KiB
aq3pnl3xfcg2h68lx7iwv7knj7992mwc-nixos-system-oenone-25.05.20241115.5e4fbfb	/nix/var/nix/profiles/system-558-link	29.3 GiB	157.5 KiB
ifrfgpyilgvigl2iary8dsxswn9313iq-nix-shell-env	/home/dan/work/resl/qualysdata/.direnv/flake-profile-a5d5b61aa8a61b7d9d765e1daf971a9a578f1cfa	2.8 GiB	72.8 KiB
2zwmk2wsxm534qyb3597p12y3bs1wim2-nix-shell-env	/data/foss/rust/serde-norway/.direnv/flake-profile-a5d5b61aa8a61b7d9d765e1daf971a9a578f1cfa	1.6 GiB	84.5 KiB
vysk2fzd96zyv1qdsd9j9x4fwdi93jpk-home-manager-generation	/home/dan/.local/state/nix/profiles/home-manager-248-link	1021.8 MiB	14.5 KiB
fjbyx7bkzfahpiklab7m1k6qzngq7d2q-home-manager-generation	/home/dan/.local/state/nix/profiles/home-manager-247-link	1017.9 MiB	14.5 KiB
z4cjvbn055bqsdgs3sjj0whs6m8gmasx-home-manager-generation	/home/dan/.local/state/nix/profiles/home-manager-246-link	1017.9 MiB	14.5 KiB

nh2 · December 12, 2024, 4:01am

I have further adjusted this to work with NixOS 24.11 (which seems to have changed the nix path-info JSON output format so the jq sort_by() didn’t work anymore.

Also padding-aligned the output, and added --extra-experimental-features nix-command to not assume a global system setting that sets this.

#!/usr/bin/env bash

# From https://discourse.nixos.org/t/show-closure-size-of-garbage-collector-roots/29118

# Collecting GC roots paths
declare -a roots
while IFS= read -r -d '' link
do
        roots+=( "$( readlink -f "$link")" )
done < <(find /nix/var/nix/gcroots/ -type l -readable -print0)

# Retreiving GC roots sizes, order by size
tmp="$(mktemp)"
trap 'rm $tmp' EXIT
nix --extra-experimental-features nix-command path-info --closure-size --json "${roots[@]}" | jq 'to_entries | sort_by(.value.closureSize) | map({ path: .key, size: .value.closureSize })' > "$tmp"

# Pretty print result
jq -r '.[] | .path + ": " + (.size | tostring)' "$tmp" | while read -r line; do
    path=$(echo "$line" | cut -d: -f1)
    bytes=$(echo "$line" | cut -d: -f2 | tr -d ' ')
    
    # Pretty-print bytes to a more appropriate human format
    readable_size=$(numfmt --to=iec-i --suffix=B --format="%8.1f" <<< "$bytes")
    
    echo "$readable_size   $path"
done

LovesTha · May 21, 2026, 1:19am

The nushell script fails on all but 1 of my nixos systems. The fix is to put ‘?’ after the column names on the select and sort-by lines:

            def gcdu [
                --processes (-p)    # also show roots from running processes
            ]: nothing -> table {
              let roots = ^nix-store --gc --print-roots 
                | parse "{root} -> {closure}"
                | where root !~ '{censored}|^/proc' or $processes
                | group-by --to-table closure 
                | rename path roots
                | update roots { get root | sort | str join "\n" }
              
              let pathinfo = ^nix path-info --closure-size --json ...($roots | get path) 
                | from json
                | select closureSize? narSize? path?
                | into filesize closureSize narSize
              
              $roots | join $pathinfo path
                | update path { path basename }
                | rename -c { path: "store path" }
                | sort-by -r closureSize?
            }

uep · May 22, 2026, 12:51am

Interesting, I don’t see this at all. This might be a change in output in recent nix (I use lix). It’s already using json for that stage, so it’s not a simple output format change and fragile regex, unlike some other similar script snippets. Unless it’s in the first stage of finding the roots.

uep · May 22, 2026, 1:50am

oh, yeah, the data has changed shape completely.

lix / older nix:

[
  {
    "ca": "fixed:r:sha256:1cii9id0k1vsa3r53k54bndyl6kxzgm1nsgj7gncgbp3j61qahlp",
    "closureSize": 47032,
    "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=",
    "narSize": 47032,
    "path": "/nix/store/01x5k4nlxcpyd85nnr0b9gm89rm8ff4x-source",
    "references": [],
    "registrationTime": 1737963696,
    "valid": true
  },
  {
   # …
  }
]

ie, a list of records, maps directly to a table in nu

new nix:

{
  "/nix/store/01x5k4nlxcpyd85nnr0b9gm89rm8ff4x-source": {
    "ca": "fixed:r:sha256:1cii9id0k1vsa3r53k54bndyl6kxzgm1nsgj7gncgbp3j61qahlp",
    "closureSize": 47032,
    "deriver": null,
    "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=",
    "narSize": 47032,
    "references": [],
    "registrationTime": 1762657409,
    "signatures": [],
    "ultimate": false
  },
  "/nix/store/0bn38nqy0h4h9j324sfvqkih2xp42lq4-home-manager-generation": {
    # …
  }
}

ie, a map of paths to records.

So you’re not getting useful data at all just by filling nulls.

the cheesy workaround is jamming | items {|p, d| $d | insert path {$p} } after the from json

A more general fix to work with both involves looking at describe to optionally do this based on the shape of the input (or a version check). I assume the formats are not considered stable/breaking.

LovesTha · May 22, 2026, 2:39am

Yeah, I don’t know what the difference between systems is, they all build from the same nixos config, so they should be the same….

And it’s been persistently this way for quite a while.

But the ? make it at least return some valid data

uep · May 22, 2026, 3:43am

Well, now you get to find out. Guesses:

one is using lix as an experiment you forgot
one is not getting updated (failing auto-updates, forgotten channel switch, …)

Also interesting, because that change just adding ?'s doesn’t work for me — the join fails because there are no paths.

But it gets more complicated. The random cppnix box I chose for the parts above is a little out of date: nix (Nix) 2.31.3

On another, with nix (Nix) 2.34.7 it emits a warning:

warning: '--json' without '--json-format' is deprecated; please specify '--json-format 1' or '--json-format 2'. This will become an error in a future release.

So, it seems there are versioned json formats after all. But even version 1 is the same as the “new” format above, version 2 adds an info object wrapper around it with versioning info. Fun.

LovesTha · May 24, 2026, 5:55am

Curiouser and curiouser.

Mine are all on lix, and have been updated enough times and mostly to the exact same versions that I’m confident it isn’t that.

Some weird dangling parts of the nix store would be more likely to me, ones that impacted all but one system for some odd reason. But I can’t find an easy way to extract out the rows that have the bad data

uep · May 24, 2026, 10:53am

try (after setting roots as per the first pipeline):

let pathinfo = ^nix path-info --closure-size --json ...($roots | get path)
    | from json
    | where (is-empty closureSize) or (is-empty narSize) or (is-empty path)

and then look at the result with $pathinfo or $pathinfo | explore or $pathinfo | save debug.json

LovesTha · May 25, 2026, 1:15am

I’m confused which rows have the nulls… I don’t think that where works.

But explore does tell me some things:

Apparently I have paths that are empty…?

LovesTha · May 25, 2026, 1:16am

Why is that a path that needs to be fetched?

What is going on here?

uep · May 25, 2026, 4:45am

not sure, but let’s take more detailed debugging to dm (or matrix)