When I run nix build for a flake I get the following prelude messages:
unpacking 'github:numtide/flake-utils/…' into the Git cache...
unpacking 'github:nix-systems/default/…' into the Git cache...
unpacking 'github:nixos/nixpkgs/…' into the Git cache...
Now, I’m running nix build in a docker container with a nix store that I’ve prepopulated with the buildDeps of the package I’m building, ideally I would want to prepopulate this “Git cache” with these flake inputs as well, and bake them into the docker image. So my question is:
a) where is this “Git cache”?
b) can I include it in my docker image in such a way as to have nix build not fetch the flake input github repos but always use this cache instead?
1 Like
Currently I’m able to work around this in an ugly way: get the ref from flake lock(rev=$(jq -r '.nodes.nixpkgs.locked.rev' flake.lock), clone the corresponding repositories into other directories (e.g. /tmp/nixpkgs) and just use nix build . --override-input nixpkgs /tmp/nixpkgs). Wonder if there are more elegant ways of doing that.
do you do this inside of nix or do you run some external script? Could you maybe write out a more complete gist? Also, I’m trying to find where exactly nix stores the cached git flake inputs, but the things in .cache/nix/gitv3 aren’t proper git repo clones, (I also feel like a whole clone would be overkill). However, maybe they are stored somewhere in the tarballs or tarball-cache (or eval-cache ?)
ok, so looking at the modification dates in .cache/nix, it would appear gitv3, gitv2 and tarballs are all no longer used, only tarball-cache and eval-cache-v5 are, and I can’t find anything looking like a nixpkgs checkout there.
You can checkout my GitHub Action definition.
In my case I have a private GitHub repo which stores my secrets (yea I know it’s bad but those arent very valuable). I want to run nixos-rebuild in Github actions but by default an action in a public repository doesn’t have access to another private one - hence here I am.
just replied to the wrong post. ping for notification
When I do nixos-rebuild switch I’m seeing “unpacking … into the Git cache” even though there is a copy of the desired nixpkgs tree available in the store.
I also have a narHash property on the flake. So my expectation/hope would be that the narHash would allow a mapping to the existing store path.
But that’s not what happens. Instead we end up with (nix build --verbose --debug):
using cache entry 'file:{"name":"flake-registry.json","store":"/nix/store","url":"https://channels.nixos.org/flake-registry.json"}' -> '{"etag":"W/\"f284424ae1a4369a3f591f28915f7e43176ccc52bdf7117ff49868055d6c522b\"","url":"https://channels.nixos.org/flake-registry.json"}', '/nix/store/pc6y12ccrlvmgfbp0rs34cbj8mbsyyvk-flake-registry.json'
looked up 'flake:nixpkgs' -> 'github:NixOS/nixpkgs/52faf482a3889b7619003c0daec593a1912fddc1?narHash=sha256-6hl6L/tRnwubHcA4pfUUtk542wn2Om%2BD4UnDhlDW9BE%3D'
did not find cache entry for 'gitRevToTreeHash:{"rev":"52faf482a3889b7619003c0daec593a1912fddc1"}'
unpacking 'github:NixOS/nixpkgs/52faf482a3889b7619003c0daec593a1912fddc1?narHash=sha256-6hl6L/tRnwubHcA4pfUUtk542wn2Om%2BD4UnDhlDW9BE%3D' into the Git cache...
downloading 'https://github.com/NixOS/nixpkgs/archive/52faf482a3889b7619003c0daec593a1912fddc1.tar.gz'...
The reason this is happening is because the flake lookup machinery lacks any knowledge of the mapping between a flake reference (even one with a specific git commit) and the source tree on disk. So it has to go away and resolve it. In my case that involves fetching the tarball. At which point I think it’s effectively getting unpacked into the store even though this is redundant.
We might be tempted to add a registry pin to pin nixpkgs to a path; but then we lose some metadata and break nix flake update (rather than going to our intended/desired pin, it ends up going to the pinned store path, which fresh machines may not have or be able to fetch).
The other reason I guess this fetch is happening is that you need all the metadata for the flake; this includes the lastModified attribute.
I do wonder though if the narHash and lastModified attribute were supplied in the flake reference if nix could avoid having to resolve the reference by fetching the tarball. That would be helpful.