Bug in hash validation for fetchFromGithub?

Hi!

While I was trying to package a few vim plugins locally, I stumbled into this weirdness. I have provided a nix flake to build two vim plugins from github. In the fetchFromGitHub function I supply the owner, repo, rev (commit id) and hash. My intuition is that fetchFromGitHub would use all of these parameters to specify the exact files and validate them against each other. However, the first time I accidentally supplied the same checksum to both plugins. When I realized this, I could not understand how nix could build it without complaining. Seems like a serious bug to me. Do you agree and are you able to reproduce it?

{
  description = "A very basic flake";

  outputs = { self, nixpkgs }: {

    packages.x86_64-linux = let pkgs = nixpkgs.legacyPackages.x86_64-linux; in {

      baleia-nvim = pkgs.vimUtils.buildVimPluginFrom2Nix {
        name = "baleia.nvim";
        src = pkgs.fetchFromGitHub {
          owner = "m00qek";
          repo = "baleia.nvim";
          rev = "00bb4af31c8c3865b735d40ebefa6c3f07b2dd16";
          hash = "sha256-jxRlIzWbnSj89032msc5w+2TVt7zVyzlxdXxiH1dQqY=";
        };
      };

      vim-arduino = pkgs.vimUtils.buildVimPluginFrom2Nix {
        name = "vim-arduino";
        src = pkgs.fetchFromGitHub {
          owner = "stevearc";
          repo = "vim-arduino";
          rev = "b2573b094ec301f2874b7ae3ec0e8806f8fb5e0e";
          # hash = "sha256-hqC28ylvBeAKBTOkjZHx6shhDjm2xO7ovKgHxzVw6IE="; # correct hash, builds without errors as expected
          hash = "sha256-jxRlIzWbnSj89032msc5w+2TVt7zVyzlxdXxiH1dQqY="; # incorrect hash but no complaints (hash is from package above and it builds the package above instead of erroring out)
           # hash = "sha256-HqC28ylvBeAKBTOkjZHx6shhDjm2xO7ovKgHxzVw6IE="; # errors out as expected
          # hash = "sha256-JxRlIzWbnSj89032msc5w+2TVt7zVyzlxdXxiH1dQqY="; # errors out as expected
        };
      };

    };
  };
}

After some experimentation, the behavior seems to be that nix only builds the plugin if it has seen the hash before. However, it does not validate the repo name, owner och rev field. Instead, it seems to find the sources that belong to the hash and build that package. So if I change the plugins hash to be the hash of the other plugin when I run nix build ".#vim-arduino" I get the baleia-nvim plugin instead of the vim-arduino plugin.

If it is a bug, where do I report it? To the nix binary github repo?

1 Like

This is Nix working as intended. All fetchers, including fetchFromGitHub, are only identified by their output hash (and their name, but that’s always source for fetchFromGitHub). owner/repo/rev/etc. do not matter when determining if an existing result can be used.

The manual has a longer explaination. Yes, it occasionally causes issues and surprising behaviour.

2 Likes

Oh, did not expect that. Any reason why the implementation is not storing the other metadata along with the hash of the contents and validates that the metadata matches the hash?

1 Like

The reason is explained in the Nix manual. It prevents unneccessary rebuilds if a file we want to download has moved to a different location.

1 Like

I think all of us have run into this issue at least once. At first, it is suprising, but it does make sense as is explained in the manual. What I always do is using fake hashes for new derivations. Those can never be satisfied.

There is lib.fakeHash from Nixpkgs, but nowadays I simply use:

hash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";

EDIT: See also What is the latest best practice to prefetch the hash? - #5 by jtojnar