Hello, I recently contributed to nixpkgs by adding a Rust binary called itm-tools but that was really just one of my beginner’s steps in learning Nix. I followed the manual and some other existing Rust apps on nixpkgs to write the script, so I learnt about the need to patch a Cargo.lock (if missing) and how to get the right cargoSha256. I’m happy to see the package works!
However, I just looked back into my code and tried to modify the code to test my knowledge. By changing the version string, I found that the package starts to fail, and it reports that the cargoSha256 has changed.
I didn’t expect this to happen. On the manual, the checksum is said to be “computed over all crate sources of this package”, so I expected that it changes only when the source and/or patches have changed. So, would anyone shed some light on what things are taken into account when cargoSha256 is getting computed? Thank you!
I don’t know a whole lot about the technical specifics. But if you changed the version, I think the dependencies are now different, so the crate sources have changed. The Cargo.toml and Cargo.lock can change as the dependencies are updated in different commits/version changes.
And it is possible that the checksum also includes the version string in some way, but I’m not sure about that.
buildRustPackage uses the fetchCargoTarball function to fetch the dependencies of the crate. This is done by fetching the dependencies using cargo vendor, applying normalization (so that irrelevant changes to the vendoring do not result in a different output), making a tarball of the vendored dependencies. fetchCargoTarball with proper arguments evaluates to a fixed-output derivation [1]. cargoSha256 is actually the hash of this fixed-output derivation.
When a package is updated to a new version, usually one or more of its dependencies change as well, resulting in changed vendored dependencies, tarball, and thus checksum.
[1] It should be mentioned that some consider this an abuse of fixed-output derivations: `buildRustPackage` needs improving or replacing - abuse of fixed-output derivations, etc. · Issue #89563 · NixOS/nixpkgs · GitHub
2 Likes
@danieldk Thank you very much for the answer! It makes sense that changes to the crate and its dependencies will modify the build and thus the checksum, and that’s actually what I expected. What I didn’t expect was just that changing the variables belonging to the crate build derivation (e.g. pname, version) would also update the output itself. (I think the last sentence by @Pacman99 said the same idea.)
And because you mentioned the name of the function fetchCargoTarball, I can now inspect how it was implemented. If I understand it correctly, there are a few things worth pointing out:
- It is a new function in nixpkgs 20.09, which replaces
fetchcargo in prior versions.
- Because of this function, a tarball containing a directory instead of just the directory’s contents (the outputs of Rust’s
cargo vendor) is now used to compute the sha256 checksum, which the cargoSha256 value passed to the buildRustPackage call is compared against.
- The name of the directory inside the tarball is by default (and often) a string concatenating
pname and version.
- Thus, as long as the derivation’s name or version string changes, the checksum gets changed. (I just found a discussion about the case here.) For example, if I changed
version from "2021-01-01" to "unstable-2021-01-01", the original cargoSha256 would be outdated.
So, before I learnt about fetchCargoTarball, I was confused to see that if I modified version without changing cargoSha256, my Rust package derivation would be fine in, say nixpkgs 20.03, but not in 20.09. However, I am still unsure if what I said above are the valid explanations, and it seems that this kind of behaviour might be debatable and object to change in the future. But regardless, I’m happy to finally get to know what’s behind the scene every time I build something on Nix! 
1 Like