Private repo access from buildGoModule

Help
1

I have a derivation using buildGoModule and a go program that depends on a private repository.

There is a similar discussion from 2019 here: git/buildGoModule + private repositories - #7 by tomberek

The setup is that my go project includes a private repo. This can be directly in the source (import (github.com/TomMD/privatething)) or indirectly in the go.mod (replace ( github.com/a/b => github.com/TomMD/b-but-private)).

Ultimately when you build, such as with a default.nix like:

{ pkgs ? import (fetchTarball {url="https://github.com/NixOS/nixpkgs/archive/e9b8b78affb3e881e33680acfd999697493fd38a.tar.gz"; sha256="0vrawckd3khx6rx1sgybppfnrgmsq6d1m7k7jyfw9nakmixxgq7a";}) {}
, gitignoreSourcePure ? pkgs.nix-gitignore.gitignoreSourcePure
}:
 pkgs.buildGoModule rec {
   pname = "hello";
   version = "1";
   src = gitignoreSourcePure [ ] ./.;
   buildInputs = with pkgs; [
     openssh
   ];
   vendorSha256 = pkgs.lib.fakeSha256;
   proxyVendor = true;
   doCheck = false;
   preCheck = ''
     export PATH="${pkgs.lib.makeBinPath [ pkgs.openssh ]}:$PATH"
   '';
   meta = with pkgs.lib; {
     description = "Hello";
     homepage = src.meta.homepage;
     platforms = platforms.linux ++ platforms.darwin;
   };
}

The result is unsurprising because it is using unauthenticated HTTP:

go: github.com/foo/bar@v0.19.1: reading github.com/TomMD/resume/go.mod at revision v0.19.1: git ls-remote -q origin in /private/tmp/nix-build-hello-1-go-modules.drv-0/go/pkg/mod/cache/vcs/f0af8278e5b83502c71826605ec31c65511d4081781b022c6c53a17aec908942: exit status 128:
        fatal: could not read Username for 'https://github.com': terminal prompts disabled

EDIT: Submitted too soon.

The linked discussion has some suggestions. We can try them. First we tell git to use ssh instead of http second we educate the sandbox about the known_hosts:

{ pkgs ? import (fetchTarball {url="https://github.com/NixOS/nixpkgs/archive/e9b8b78affb3e881e33680acfd999697493fd38a.tar.gz"; sha256="0vrawckd3khx6rx1sgybppfnrgmsq6d1m7k7jyfw9nakmixxgq7a";}) {}
, gitignoreSourcePure ? pkgs.nix-gitignore.gitignoreSourcePure
}:
 pkgs.buildGoModule rec {
   pname = "hello";
   version = "1";
   src = gitignoreSourcePure [ ] ./.;
   buildInputs = with pkgs; [
     openssh
   ];
   vendorSha256 = pkgs.lib.fakeSha256;
   proxyVendor = false;
   overrideModAttrs = old: {
         preBuild = ''
         export HOME=$(pwd)
         cat <<EOF > $HOME/.gitconfig
         [url "git@github.com:TomMD"]
             insteadOf = "https://github.com/TomMD"
         EOF
         mkdir ~/.ssh
         chmod 700 ~/.ssh
         cat <<EOF >~/.ssh/known_hosts
         github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl
         EOF
         chmod 644 ~/.ssh/known_hosts
         export GIT_SSH_COMMAND="${pkgs.openssh}/bin/ssh -o UserKnownHostsFile=$HOME/.ssh/known_hosts"
         '';
       impureEnvVars = pkgs.lib.fetchers.proxyImpureEnvVars ++ [ "SSH_AUTH_SOCK" ];
   };
   doCheck = false;

   meta = with pkgs.lib; {
     description = "Hello";
     homepage = src.meta.homepage;
     platforms = platforms.linux ++ platforms.darwin;
   };
}

This gets us much of the way, ssh doesn’t object to the host SSH key and it tries to pull from the correct repo. nix-build fails with:

go: errors parsing go.mod:
/private/tmp/nix-build-hello-1-go-modules.drv-0/go-hello-world/go.mod:6:2: replace github.com/TomMD/go-git: git ls-remote -q origin in /private/tmp/nix-build-hello-1-go-modules.drv-0/go/pkg/mod/cache/vcs/24658a01e857650bb9f5c0bd2b11fcf9feb3612f395da37911296c9feb0445d5: exit status 128:
        git@github.com: Permission denied (publickey).
        fatal: Could not read from remote repository.

Now in the prior (linked) conversation this is where ssh-agent was hinted at with an impure of SSH_AUTH_SOCK. That doesn’t work and certainly shouldn’t since the socket file isn’t available in the build sandbox. What’s the solution here?

2

Without knowing the intricacies of go’s download process, if it’s available via https, could a netrc file be used instead? The other discussion seems to require ssh, which is more tricky.

3

I can’t get .netrc to work with nix-build + go. After perhaps 45 minutes I gave up on that avenue, but if someone else has success then a small write up would be appreciated. Till then, the “best” solution is probably still to perform go mod vendor outside of nix.

1 Like