I’m trying to compute the build time closure of a derivation, outside of Nix. Slightly misleading title, because I think I’ve done most of the work, just ended up stuck on some details
I naively assumed parsing a derivation and then recursively gathering all
inputSrcs would suffice, but that doesn’t seem to hold - at least not for my
nixos-system derivation. Using this strategy I can discover 7752 out of 7790 paths compared to
nix-store -qR of my system derivation - so there are 38 missing.
The difference seems to stem from the store path for
modules.xml (nixpkgs src) which is generated using
builtins.toFile. Importantly, this file contains XML includes, which explains the dependency:
$ nix why-depends /nix/store/yb71ygc8dpxnkp64vdar1wazjf5wjbn7-modules.xml /nix/store/7l34sk54sjdlh7xcivrmj8gs312xfirg-pict-rs.xml /nix/store/yb71ygc8dpxnkp64vdar1wazjf5wjbn7-modules.xml ╚═══/: …/>.<xi:include href="/nix/store/7l34sk54sjdlh7xcivrmj8gs312xfirg-pict-rs.xml" />.<xi:include hre… => /nix/store/7l34sk54sjdlh7xcivrmj8gs312xfirg-pict-rs.xml
If I query the nix-store for requisites of
modules.xml I find all 38 missing paths:
$ nix-store -qR /nix/store/yb71ygc8dpxnkp64vdar1wazjf5wjbn7-modules.xml /nix/store/07fi8dm3inmsbxhnydzjl0jshw8346hn-trezord.xml /nix/store/0g8yl6sbi4knqfi8n48f06v2vyx31770-yggdrasil.xml /nix/store/0nxlsrqhghky7v0shfsld98mlcfbvqrx-borgbackup.xml ...
Reading Eelco’s PhD thesis to figure out whether this is intended leads me to believe it is (because by definition all recursive references are part of a closure and they don’t necessarily have to be in any derivation inputSrcs), but I remain puzzled by the fact that this is the only occurrence (in my system closure) of a dependency not reachable by only parsing derivations - it requires outside knowledge of the dependency (e.g. at evaluation time, which is how I believe Nix handles it).
Does this make sense?
Am I missing anything that does imply
modules.xml should be built from a derivation instead (which would solve my problem at least for my current system derivation)?
What’s the easiest way to figure out these missing dependencies? I managed to come up with either scanning all inputs for unknown store paths (slow) or querying the Nix store database (which keeps a table of references), just to get the last handful of paths…