buildGoModule - how to get vendorSha256

I am trying to update some Go module which uses buildGoModule. How exactly can I find out vendorSha256 (ahead of time)? Sure, I can try to install with wrong hash and then fill in the expected one for the next run, but I was hoping for something more like nix-prefetch-url which I use to obtain sha256 for fetchFromGitHub.

vendorSha256 is the sha256 of the vendored dependencies

Ok, but what does that mean actually? If I have go.sum is there some easy way to calculate it? I want to do all this updates semi-automatically. That shell script shouldn’t really need to do any kind of installation (except download and unpack), if the script was simple to understand that would be a huge plus :slight_smile:

Thanks.

1 Like

It means that it is the sha on the contents of a hypothetically vendored sub-folder if all dependencies of go.sum would have been vendored there.

Since this folder is not actually checked in into version control, it has to be fetched from remote sources during build. To ensure it’s integrity after the fetch, nix’ go build machinery requires this checksum.

Having said this, if upstream already vendors the dependencies (despite of using go modules), then this is not required (since the reop’s own sha already ensures content integrity).

Furthermore, I have come to the conclusion that nix does not tend to trust the checksums of third party tools. The principal reason might be somewhere in between not invented here and maintaining full source control.

Hence, it is probably not possible to calculate the consolidated vendor sha from go.sum or from anything else than the fully instantiated sub folder (which requires fetching).

I personally think, there is room for improvement on how things currently work (either by proxying the language specific integrity mechanisms like go.sum or directly trust them).

Here’s a real world example: nixpkgs-wayland/update.sh at c9c4a80715557caf57f403038a267f3c20859424 · colemickens/nixpkgs-wayland · GitHub

That’s how nixpkgs-wayland pre-calculates vendorSha256 for golang packages in my overlay. See also: Calculate cargoSha256? · Issue #6 · msteen/nix-prefetch · GitHub which is where I learned about this, before adapting it for my update.sh script.

In practice, if you have a derivation for your package, you can use nix-prefetch for your needs. And I’m having a hard time imagining why you’d want the vendorSha256 if not to also use with a derivation. :slight_smile:

Hope it helps!

2 Likes

Yeah, I mean with go.sum the stuff is quite deterministic. It would be great if you could at least opt-out from the vendorSha256 requirement.

Thanks also colemickens for the code sample. Ok, your go pkg has an go-modules. But I don’t really understand the stuff you pass to nix-prefetch. This is a function that accepts sha256 as a parameter? How/where does that come from? Sorry for the stupid question(s) but I’d like to understand the stuff, not just blindly copy-paste.

1 Like

The decisive hint is dug in here:

This works because nix-prefetch support any fetcher function, even those defined on-the-fly, as long as it takes the hash as an argument and produces a fixed-output derivation. So I leveraged that to create a fetcher function specialized to just fetch pet.go-modules .

https://github.com/msteen/nix-prefetch/issues/3#issuecomment-497960452

Great thanks for providing the solution!

But what if the package is flake-based?

1 Like