fetchFromGitHub supports GitHub enterprise via the private flag, and it supports submodules via fetchSubmodules, but for some reason it doesn’t support this combination (there’s an assertion for it). I can’t find any explanation for why it doesn’t support submodules with private. I tracked it down to commit b71b7ee62 by @shlevy which added support for private but it’s unclear why this can’t support submodules.
Does anyone know what this limitation is for?
mostly related to the pkgs.fetchgit vs builtins.fetchgit changes that happened. There used to be a fetchgitPrivate fetcher which may have been the reason for the assertion (the original pkgs.fetchgit couldn’t handle credentials IIRC).
Either way, this can likely be updated, as fetchGitHub will use the builtins.fetchgit fetcher for the fetchSubmodules option anyway.
fetchgit is still a derivation, the builtin is fetchGit (which is rather confusing). fetchFromGitHub uses fetchgit, not fetchGit. I didn’t realize there was a separate fetchgitPrivate previously; if fetchgit doesn’t have any way of specifying credentials, that would explain it.
Either way, looking into this more, fetchFromGitHub is using a netrc file that it constructs out of env vars for fetching from private repos. fetchgit doesn’t seem to support credentials at all and I’m guessing fetchGit doesn’t support the netrc approach. It’s not clear to me how fetchGit even handles credentials; there’s an example in the manual labeled Fetching a private repository over SSH that looks like
builtins.fetchGit {
url = "git@github.com:my-secret/repository.git";
ref = "master";
rev = "adab8b916a45068c044658c4158d81878f9ed1c3";
}
but there’s no credentials here. This implies it’s relying on git’s ssh credential store, but I don’t know what is supported here when evaluating from Nix. Surely it’s not just using whatever ssh-agent info is in my environment? And what if I want to fetch over HTTP instead of SSH?
It also occurs to me that fetchGit doesn’t have any way to fetch submodules, and presumably leaves the .git intact, so I’d have to write my own wrapper fetcher just to fetch submodules as well.
From my experience builtin fetchGit is executed outside of sandbox, so it uses however your git is configured in your system (netrc, ssh private key)
That’s correct. All the builtins.fetch* functions are un-sandboxed so they can use the user’s regular credentials without any special hacks.
Also worth mentioning that since yesterday builtins.fetchGit has submodule support.
How does building fetchGit with credentials supposed to work when using a nix-daemon? Is there a nice way to pass a user’s ssh-agent? I’ve used socat and other hacks to expose it to the nixbld users/group, but not sure if there is a cleaner way. Using “-Issh-auth-sock” only works if the file is already readable by the nixbld users.
@edolstra I’ve run into an interesting case where I want the bind-mounting of the sandbox, but with the sandbox off. So if I want to access something via an SSH auth-sock, there are options in nix to bind mount the socket to a standard location in the build environment, but then I can’t access network. If I turn sandboxing off, I lose the ability to bind mount.