What is the latest best practice to prefetch the hash?

Help
1

It took jumping through a few hoops to update TreeSheets’ Nix expression to use the latest release.

  1. Remembered that changing the sha256 attribute to all zeroes will yield the correct value when running nix-shell,

    src = fetchFromGitHub {
  owner = "aardappel";
  repo = "treesheets";
  rev = "4778c343ac78a3b3ccdaf079187b53d8cd64e235";
# sha256 = "oXgOvvRoZpueEeWnD3jsc6y5RIAzkXzLeEe7BSErBpw=";
  sha256 = "00000000000000000000000000000000000000000000";
};

    but that didn’t work out:

    $ nix-shell -I nixpkgs=~/clones/nixpkgs/ -v -p treesheets
error: invalid base-64 hash '00000000000000000000000000000000000000000000'

  2. Looked at the Update a package NixOS wiki article that reminded me of nix-prefetch-url,

    $ nix-prefetch-url https://github.com/aardappel/treesheets/archive/4778c343ac78a3b3ccdaf079187b53d8cd64e235.zip
path is '/nix/store/ljczp58hwy2l5sadl01a002461l22q5m-4778c343ac78a3b3ccdaf079187b53d8cd64e235.zip'
0sd4823m8a9sglz9hyknb8x7gpajad4s52gb9ljvl7d7b1dpgg9m

    updated sha256 with the result,

    src = fetchFromGitHub {
  owner = "aardappel";
  repo = "treesheets";
  rev = "4778c343ac78a3b3ccdaf079187b53d8cd64e235";
# sha256 = "oXgOvvRoZpueEeWnD3jsc6y5RIAzkXzLeEe7BSErBpw=";
# sha256 = "00000000000000000000000000000000000000000000";
  sha256 = "0sd4823m8a9sglz9hyknb8x7gpajad4s52gb9ljvl7d7b1dpgg9m";
};

    but no joy:

    $ nix-shell -I nixpkgs=~/clones/nixpkgs/ -v -p treesheets
error: hash mismatch in fixed-output derivation '/nix/store/r59nywpis5zyws6hyy5q2km7qfws9nlw-source.drv':
        specified: sha256-Nb13W1inHbolTeuJoklTUt13Olp2epg+fTopVIdApGk=
            got:    sha256-UyltzT9B+7/hME7famQa/XgrDPaNw3apwchKgxwscOo=

  3. Gave the got: error result a try,

      src = fetchFromGitHub {
    owner = "aardappel";
    repo = "treesheets";
    rev = "4778c343ac78a3b3ccdaf079187b53d8cd64e235";
  # sha256 = "oXgOvvRoZpueEeWnD3jsc6y5RIAzkXzLeEe7BSErBpw=";
  # sha256 = "00000000000000000000000000000000000000000000";
  # sha256 = "0sd4823m8a9sglz9hyknb8x7gpajad4s52gb9ljvl7d7b1dpgg9m";
    sha256 = "sha256-UyltzT9B+7/hME7famQa/XgrDPaNw3apwchKgxwscOo=";
  };

    and this time it worked, but never saw this version of the hash before.

Just when I finished typing all this did I remember about nix-prefetch-url --unpack,

$ nix-prefetch-url --unpack https://github.com/aardappel/treesheets/archive/4778c343ac78a3b3ccdaf079187b53d8cd64e235.zip
path is '/nix/store/bdwd3mcdfimd47ib15pqznf8hg4pwv8w4778c343ac78a3b3ccdaf079187b53d8cd64e235.zip'
1skh5hf86jn8q6lpdhwdyq62ny7x39j6mpsf63hvzys17z6nsaak

which works also, but then the format is totally different.

2

lib.fakeHash exists to give the correct format of hash, but a useless value (I think it’s all zeros), so you could use that to start the trust-on-first-use loop. (You can also just change one character in it to something that’s clearly valid in that location.)

There are actually several forms of hash accepted by nix, so the “change to all zeros” only works if the format you started with was the right type for that (this one wasn’t).

Also, that format is an SRI hash. See https://www.srihash.org/.

1 Like
3

I use the flake to lock the hashes of any Nix expression code, and I use nvfetcher to keep packages sources up to date. I dunno about best practice but it is definitely more convenient than doing this manually.

1 Like
4

A hash is a binary string but since Nix files are plain text, using the hash as is would not be very convenient. For that reason a hash is typically represented as a number. To make the number shorter bases other than the common base-10 is used. Nix supports the following, demonstrated on SHA-256 hash with the numerical value 0:

  • base-64: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
  • base-32: 0000000000000000000000000000000000000000000000000000
  • base-16: 0000000000000000000000000000000000000000000000000000000000000000

Nix uses the length of the hash to determine the format. Since you replaced the base-64 hash with zeroes it was considered a base-64 format. In the encoding, each digit represents 6 bits so you need ⌈256/6⌉=43 characters. Since RFC 4648 wants the base-64 numbers to have lengths divisible by four, = is required as a padding. But you cannot really change it to 0, as that would give you 44*6=264 bits.

2 Likes
5

Right, fetchFromGitHub hashes the extracted source directory, while nix-prefetch-url hashes the downloaded file by default. As you discovered you will need to use --unpack when you want to use the hash for fetchzip or fetchFromGitHub, which often uses fetchzip internally.

You can also just set the sha256 attribute to an empty string and Nix will tell you the correct one.

This is base-32 encoded hash, which sha256 attribute supports just as well.

In this case, there is an update script so you can just run the following in your Nixpkgs checkout:

nix-shell maintainers/scripts/update.nix --argstr commit true --argstr path treesheets

nixpkgs-update bot also does that periodically, someone just needs to notice the PR and merge it: treesheets: unstable-2022-03-12 -> unstable-2022-09-26 by r-ryantm · Pull Request #190656 · NixOS/nixpkgs · GitHub.

1 Like
6

Here is the relevant section in the Nixpkgs manual (source).

@toraritte Would be great if you could update that with the insights found here.

@jtojnar Maybe you can yourself add the bits you revealed here to the manual?

It feels like it would be multiple days of work to put all the correct information at the right level of detail into the correct places, but having it in one of the manuals at all would already be a good thing.

1 Like
7

Besides lib.fakeHash, you can also leave the string empty (i.e. ""), and most functions in nixpkgs will then automatically substitute a hash. This takes the least typing, and doesn’t require a lib in scope if you’re working on something in a callPackage, so I tend to use that feature a lot when experimenting.

It does not work for fetchpatch, notably, but fetchFromGitHub does it.

1 Like