`builtins.fetchurl`vs `builtins.fetchTree { type = "file"; }` hashes

This was something I ran into today that deals which some implementation detail minutiae; but it may be of interest to some of you.

Basically I was trying to find out why builtins.fetchTree { type = "file"; narHash = "..."; } and builtins.fetchurl { sha256 = "..."; } want different hashes.

This is probably really obvious to some of you who have been at this for years ( unless you’re like me and just forgot ); but for others it may be puzzling.

  let
    url     = "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz";
    narHash = "sha256-fn2qMkL7ePPYQyW/x9nvDOl05BDrC7VsfvyfW0xkQyE=";
    ftree = builtins.fetchTree { type = "file"; inherit url narHash; };
    # {
    #   narHash = "sha256-fn2qMkL7ePPYQyW/x9nvDOl05BDrC7VsfvyfW0xkQyE=";
    #   outPath = "/nix/store/64icjs49plygl816k6hn23vcmi5ccgsn-source";
    # }

    # Fails: sha256 mismatch...
    # furl_bad0 = builtins.fetchurl { inherit url; sha256 = narHash; };

    # Try matching the same outfile name ( which shouldn't matter )
    # Fails: sha256 mismatch...
    # furl_bad1 = builtins.fetchurl { inherit url; sha256 = narHash; name = "source"; };

    caHash = builtins.hashFile "sha256" ftree.outPath;
    # => "6a087ac9e5702a0c9d60fbcd48696012646ec8df1491dea472b150e79fcaf804"

    # Use content addressed hash and voila
    furl = builtins.fetchurl { inherit url; sha256 = caHash; }
    # => "/nix/store/n4alpyll6q2x61j79s5na7val380cx7g-lodash-4.17.21.tgz"
  in {
    inherit furl ftree;
  }

Okay so one seems to be CA and the other isn’t, which I didn’t realize mattered for “fixed output derivations” ( which I guess aren’t CA ??? ).

Also just for context:

  • yes I am 100% positive the files are identical.
  • yes I checked the full file stat .
  • yes I diffed the tarballs.
  • yes I am positive that the difference in filename doesn’t matter.

Some nix path-info output reveals some more info:

  sh> URL="https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz";
  sh> FTOP="$( nix eval --raw --impure --expr '( builtins.fetchTree { type = \"file\"; url = \"$URL\"; } ).outPath'; )";
  sh> FUOP="$( nix eval --raw --impure --expr 'builtins.fetchurl { url = \"$URL\"; }'; )";
  sh> nix path-info --json "$FUOP"|jq;
  [
    {
      "path": "/nix/store/n4alpyll6q2x61j79s5na7val380cx7g-lodash-4.17.21.tgz",
      "narHash": "sha256-fn2qMkL7ePPYQyW/x9nvDOl05BDrC7VsfvyfW0xkQyE=",
      "narSize": 319080,
      "references": [],
      "ca": "fixed:sha256:017qragyfl5ifajdx48lvz46wr0jc1llikgvc2fhqakhwp4pl23a",
      "deriver": "/nix/store/ccvfy7dilj5zvrz0vl4iria8ylmqzmb3-lodash-4.17.21.tgz.drv",
      "registrationTime": 1666118258,
      "ultimate": true
    }
  ]
  sh> nix path-info --json "$FTOP"|jq;
  [
    {
      "path": "/nix/store/64icjs49plygl816k6hn23vcmi5ccgsn-source",
      "narHash": "sha256-fn2qMkL7ePPYQyW/x9nvDOl05BDrC7VsfvyfW0xkQyE=",
      "narSize": 319080,
      "references": [],
      "ca": "fixed:sha256:017qragyfl5ifajdx48lvz46wr0jc1llikgvc2fhqakhwp4pl23a",
      "registrationTime": 1665502179
    }
  ]

The difference to notice is that one has deriver ( internal builtins:fetchurl derivation generator ), and some mysterious ultimate arg.

In any case, what we lerned is that builtins.fetchurl wants you to specify the CA hash, while builtins.fetchTree wants the narHash .

1 Like

Stuff with “CA” in name is new-ish and not really stable AFAIK; here I consider it a red herring.

I recommend docs around outputHashMode: Advanced Attributes

Hosted by Flying Circus.