I am misunderstanding a fundamental thing here that I would like to ask about.
If I am getting something from the binary cache, why does it seem like my nix store is populated with many of its build-time dependencies as well? (since my machine did not have to build anything)
So let’s say for example I am trying to “build” ranger right now. I am getting this output:
these paths will be fetched (5.10 MiB download, 20.06 MiB unpacked):
/nix/store/1wc15pic61ff58d872nxm9am7kc75xlf-openjpeg-2.3.1
/nix/store/3xkbwd9vznd0lglqzhrxwsvqv0nbc3b4-python3.8-olefile-0.46
/nix/store/8f7klvnrvzkksj2vx53plqjgqw35rn3w-w3m-0.5.3+git20190105
/nix/store/9g2ki25jcf39f3q2x3mbnnbmh6fbwpdj-giflib-5.2.1
/nix/store/b778c1vnsbwkk0kc5gslz8yzm628qyrs-lcms2-2.11
/nix/store/cz6dhnnvvizh205k5bb993sc8jnpvx38-gpm-1.20.7
/nix/store/d9di2grdykar2sdmvj2qdv1hs5sjhfn5-libimagequant-2.12.6
/nix/store/hjqk6b67z51j8fn998hk6gkgqfx5drj2-man-db-2.9.3
/nix/store/jhj4hw47din4alwxa2n4bj1rg81v9my1-libpipeline-1.5.3
/nix/store/m4da8ksgkp2vn8a0rgj61w6g7rmw2bkx-boehm-gc-8.0.4
/nix/store/m5pxwsl8nj55j7c00q6kmpx91adcz1nb-python3.8-Pillow-7.2.0
/nix/store/mv33vz2d4425kk4da7bgy0wv9anwxpfk-ranger-1.9.3
/nix/store/ngh6l2z1xkn617j12xd15f59yzbxfc38-imlib2-1.7.0
/nix/store/pzg1q8jiv7i6d1nridy11fpyca6qnmpl-db-5.3.28
/nix/store/rl09wq1lbkfndlpwgl5s70350b37k72l-libjpeg-turbo-2.0.4
/nix/store/vf2fsb6aprkyasr84354glpx7f7sqz1v-libid3tag-0.15.1b
/nix/store/x5jxqf6va7sqxbx1g01zsgj8id0h1qjf-libtiff-4.1.0
/nix/store/z4pv73ba1wp387rz81g0fkab32zrs1y6-libwebp-1.0.3
copying path '/nix/store/3xkbwd9vznd0lglqzhrxwsvqv0nbc3b4-python3.8-olefile-0.46' from 'https://cache.nixos.org'...
copying path '/nix/store/m4da8ksgkp2vn8a0rgj61w6g7rmw2bkx-boehm-gc-8.0.4' from 'https://cache.nixos.org'...
copying path '/nix/store/pzg1q8jiv7i6d1nridy11fpyca6qnmpl-db-5.3.28' from 'https://cache.nixos.org'...
copying path '/nix/store/9g2ki25jcf39f3q2x3mbnnbmh6fbwpdj-giflib-5.2.1' from 'https://cache.nixos.org'...
copying path '/nix/store/cz6dhnnvvizh205k5bb993sc8jnpvx38-gpm-1.20.7' from 'https://cache.nixos.org'...
copying path '/nix/store/b778c1vnsbwkk0kc5gslz8yzm628qyrs-lcms2-2.11' from 'https://cache.nixos.org'...
copying path '/nix/store/vf2fsb6aprkyasr84354glpx7f7sqz1v-libid3tag-0.15.1b' from 'https://cache.nixos.org'...
copying path '/nix/store/d9di2grdykar2sdmvj2qdv1hs5sjhfn5-libimagequant-2.12.6' from 'https://cache.nixos.org'...
copying path '/nix/store/rl09wq1lbkfndlpwgl5s70350b37k72l-libjpeg-turbo-2.0.4' from 'https://cache.nixos.org'...
copying path '/nix/store/jhj4hw47din4alwxa2n4bj1rg81v9my1-libpipeline-1.5.3' from 'https://cache.nixos.org'...
copying path '/nix/store/x5jxqf6va7sqxbx1g01zsgj8id0h1qjf-libtiff-4.1.0' from 'https://cache.nixos.org'...
copying path '/nix/store/hjqk6b67z51j8fn998hk6gkgqfx5drj2-man-db-2.9.3' from 'https://cache.nixos.org'...
copying path '/nix/store/z4pv73ba1wp387rz81g0fkab32zrs1y6-libwebp-1.0.3' from 'https://cache.nixos.org'...
copying path '/nix/store/1wc15pic61ff58d872nxm9am7kc75xlf-openjpeg-2.3.1' from 'https://cache.nixos.org'...
copying path '/nix/store/ngh6l2z1xkn617j12xd15f59yzbxfc38-imlib2-1.7.0' from 'https://cache.nixos.org'...
copying path '/nix/store/m5pxwsl8nj55j7c00q6kmpx91adcz1nb-python3.8-Pillow-7.2.0' from 'https://cache.nixos.org'...
copying path '/nix/store/8f7klvnrvzkksj2vx53plqjgqw35rn3w-w3m-0.5.3+git20190105' from 'https://cache.nixos.org'...
copying path '/nix/store/mv33vz2d4425kk4da7bgy0wv9anwxpfk-ranger-1.9.3' from 'https://cache.nixos.org'...
/nix/store/mv33vz2d4425kk4da7bgy0wv9anwxpfk-ranger-1.9.3
That’s 18 dependencies fetched, all needed at run-time.
Let’s check out the build-time closure of this thing.
$ nix-store -qd /nix/store/mv33vz2d4425kk4da7bgy0wv9anwxpfk-ranger-1.9.3
/nix/store/rxrxy66izcr7dmbcblzqyrdl41d05d0l-ranger-1.9.3.drv
$ nix-store -qR /nix/store/rxrxy66izcr7dmbcblzqyrdl41d05d0l-ranger-1.9.3.drv | wc -l
808
Much bigger, fair enough. I did not have to fetch these 800 paths, is that correct?
If I add the --include-outputs flag, however I am getting stuff from my nix store!
Documentation says about --include-outputs:
--include-outputs
Also include the output path of store derivations, and their closures.
This query can be used to implement various kinds of deployment. A source deployment is obtained by distributing the closure of a store
derivation. A binary deployment is obtained by distributing the closure of an output path. A cache deployment (combined source/binary
deployment, including binaries of build-time-only dependencies) is obtained by distributing the closure of a store derivation and specifying
the option --include-outputs.
$ nix-store -qR --include-outputs /nix/store/rxrxy66izcr7dmbcblzqyrdl41d05d0l-ranger-1.9.3.drv | wc -l
1104
That’s 1104-808 = 296 output paths.
Doing the following I can assure all paths actually exist in the store
$ nix-store -qR --include-outputs /nix/store/rxrxy66izcr7dmbcblzqyrdl41d05d0l-ranger-1.9.3.drv | xargs ls
... (long output) ...
$ echo $?
0
If one of the paths did not exist ls would have returned a non-zero error code.
I assume that the fact that only 296 out of 808 the derivation paths had their
output included by --include-outputs is because the other derivations’ output
does not exist in my nix-store? Is that assumption correct?
And how did these 296 outputs end up on my system? Is it feasible that all of
them were there in the first place in order to build something in the past that
was not cached?
I also repeated this with a random derivation from nixpkgs that seems as unrelated
to my system as possible, such as fixDarwinDylibNames (as I’m using Linux).
When I did nix-store --deriver I got unknown deriver so surely it was just a single
binary deployment no?
After nix-instantiateing it to get a derivation file and checking the numbers
$ nix-store -qR /nix/store/qja24prvvlyawzidsfhsxh7kmmfy858i-hook.drv | wc -l
246
$ nix-store -qR --include-outputs /nix/store/qja24prvvlyawzidsfhsxh7kmmfy858i-hook.drv | wc -l
349
349-246 = 103 out of 246 of the build-time dependencies exist on my system.
Perhaps these numbers all make sense, I am afraid I am not intimate enough with
this to grasp whether or not I am misunderstanding something or not.
Any help is greatly appreciated!
Thanks!