Question about fetchers and TLS

I’ve been learning about Nix and Nixpkgs and experimenting with various things recently, including using stuff like nix-prefetch-url and various fetchers. I came across some interesting info on the subject here (including the comment after it):

Looking at the relevant code here (as far as I can understand it anyway), it does appear to be the case that fetchurl is not checking the certificate:

So, I have only a pretty basic understanding of some of the relevant security concepts, so excuse me if this is a dumb question, but is this not a security problem when the validity of the certificates is not being verified? Or are the sha256 checksums when fetching source code adequate to make sure you are really downloading the intended file? If I understand correctly, then it makes sense to me that running nix-prefetch-url or fetchurl without a hash would check the certificate because you want to be sure you are connecting to the real server when you are first downloading the files to actually compute the hash that will be used in your nix expressions. That being said, is it really fine to not check the validity of the certificate in other cases when you are running the fetcher with the expected hash included, or is this a security risk?

In addition, would the fact that the fetcher is not checking the validity of the certificate mean the connection is not actually encrypted even if the link uses HTTPS protocol? Sorry if this is a dumb question, but it is unclear to me. I appreciate any info, thanks.

It is encrypted, but the connection is made to a potentially untrustworthy target, which kind of defies the purpose. A MITM attack, as could be carried out by a mischievous ISP or such, could intercept your connection and know what you are trying to download, as well as get any infortmation you’re sending with the fetchurl (which is not much, you’d probably get most of it from just the URL, which is sent plaintext either way).

These attacks are still possible if you do check the certificate, but take more dedicated effort.

That’s probably the extent of the security risk here, though. As long as you’re not using sha-1 or another known-insecure checksum, it’d take heat-death-of-the-universe levels of computation to send you a file that’s different from the one you’re expecting and actually have nix use it.

That said, some packages do use sha-1 (mostly those backed by incompetent upstreams ala Mojang). Perhaps it’s worth looking into deprecating use of dated checksums in fetchurl.

Note that due to the https trust model, you’re at the mercy of large crime organizations (including the NSA) anyway. It’s known that many root certificates have leaked, and believed that a lot of them continue to leak constantly.

This is impossible, by the way. Nix won’t let you use it without a hash, so not checking the certificate is definitely defensible, though I’m more worried about bad hash algorithms now than I was before.

I wonder if this is a worthwhile optimization or just avoiding bootstrap issues?

Edit: Hrmm, I’d just remembered http-basic auth… This may need some work.

1 Like

Thank you for the information. I am also wondering, do the other fetchers mentioned here behave in the same way as far as TLS goes?

To my knowledge, all of them internally call through to fetchurl. The (most?) definitions live in pkgs/build-support/fetch* if you’re interested.

Sorry if this is a noob question, but I don’t understand how the sha256sum of a derivation is… derived / checked, how it is used when building from source. This is germane to the discussion of TLS (to me), so i’m adding my reply in the same topic.

I’m trying to nix build an older version of hledger (1.22.2) , and the binary cache no longer has the tarball blob, so nix build needs to fetch the source code tarball from hackage. I noticed all this because all outgoing HTTP is forbidden on my home network.

error checking the existence of
curl: (28) Failed to connect to port 80: Connection timed out


The mirrors for hackage are all HTTP

  # Hackage mirrors
  hackage = [

There is a sha256sum in hackage-packages.nix:

$ grep -rA14 hledger_1_22_2
pkgs/development/haskell-modules/hackage-packages.nix:  "hledger_1_22_2" = callPackage
pkgs/development/haskell-modules/hackage-packages.nix-    ({ mkDerivation, aeson, ansi-terminal, base, base-compat-batteries
pkgs/development/haskell-modules/hackage-packages.nix-     , bytestring, cmdargs, containers, data-default, Decimal, Diff
pkgs/development/haskell-modules/hackage-packages.nix-     , directory, extra, filepath, githash, hashable, haskeline
pkgs/development/haskell-modules/hackage-packages.nix-     , hledger-lib, lucid, math-functions, megaparsec, mtl, old-time
pkgs/development/haskell-modules/hackage-packages.nix-     , process, regex-tdfa, safe, shakespeare, split, tabular, tasty
pkgs/development/haskell-modules/hackage-packages.nix-     , temporary, terminfo, text, time, timeit, transformers
pkgs/development/haskell-modules/hackage-packages.nix-     , unordered-containers, utf8-string, utility-ht, wizards
pkgs/development/haskell-modules/hackage-packages.nix-     }:
pkgs/development/haskell-modules/hackage-packages.nix-     mkDerivation {
pkgs/development/haskell-modules/hackage-packages.nix-       pname = "hledger";
pkgs/development/haskell-modules/hackage-packages.nix-       version = "1.22.2";
pkgs/development/haskell-modules/hackage-packages.nix-       sha256 = "1g1v56fxgs7ya8yl22brwgrs49a50kd77k8ad8m8l5cnlnviqb3g";
pkgs/development/haskell-modules/hackage-packages.nix-       isLibrary = true;
pkgs/development/haskell-modules/hackage-packages.nix-       isExecutable = true;

I can see that the sha256sum in hackage-packages.nix matches the hashed blob that is being retrieved from

Now, because the tarball is no longer on (I checked for it manually and it is 404 not found, which makes sense since 1.22.2 was an older version). So the builder will most likely move on to downloading from

Now the actual problem is that when I manually download the tarball and compute the sha256sum, I get


which is different from the sha256sum in hackage-packages.nix

Why are the checksums different? Any pointer to reference material is welcome, I MUST understand the behaviour of nix build commands when issuing build commands that fetch source tarballs over HTTP.

The sha in the hledger expression probably is “recursive”, and refers to the unpacked content of the tarball.

It is equivalent with unpacking the tarball to a store path, including normalisation of permissions, owners, dates, etc. Then this storepath is packed to a NAR, and this NARs sha256 is what you see in the expression.

Most of the hashes are built like that. IIRC only requireFile and fetchurl use “flat” hashes.

Thanks for clarifying this, I don’t know which kind of checksum is used for hledger, so I have to dig deeper, at least this one time, to understand what is going on.

Do you happen to know where the logic for performing a ‘recursive’ checksum computation is located?
As well as the location where the checksum check is actually performed? The nixpkgs codebase size is a little intimidating.

Answering my own question, it looks like the sha256sum in the hackage-packages.nix file matches what I get from calling nix-prefetch-url without the --unpack option

$ torsocks nix-prefetch-url
path is '/nix/store/qfrxzcyj2a7a8l873a36jqxswy0c9189-hledger-1.22.2.tar.gz'

$ torsocks nix-prefetch-url --unpack
path is '/nix/store/969wsd5ir5m1cjy375374fd388jhz7bs-hledger-1.22.2.tar.gz'

But nix-store --add-ing the downloaded file does not result in the same store file, so I definitely don’t have the full picture yet.

$ nix-store --add hledger-1.22.2.tar.gz 

I’ll look around until I understand what is going on. I care mostly that the checksum is indeed checked, and about finding the code that actually does that. For my own peace of mind.

I gained some more understanding on how to check hashes on my own.

I first fetch the hackage tarball and make sure I have exactly that copy locally,
with the same base32 ‘flat’ sha256 as that in hackage-packages.nix

$ nix-prefetch-url
path is '/nix/store/qfrxzcyj2a7a8l873a36jqxswy0c9189-hledger-1.22.2.tar.gz'

$ nix-prefetch-url file://<(cat from-hackage/hledger-1.22.2.tar.gz)
path is '/nix/store/4ckzvvjh43bjrdbfyx6hm5b53g740x88-63'

Then I build the tarball on my own from the hledger project:

(in the hledger 1.22.2 git repo clone)
$ stack --nix --resolver lts-18 sdist

Then I checked in the wrong way at first whether the tarball from hackage matched the tarball from sdist

$ nix-prefetch-url file://<(cat from-sdist/hledger-1.22.2.tar.gz)
path is '/nix/store/fvvp66ids9k9pw5r3inw8cwq68ah6jw3-63'

The hash differs because there are differences in file attributes (file creation times, etc).

To check the right way, based on what I found on Nix Hash - NixOS Wiki,
the --unpack option is important. It must be used for both the tarball from hackage and
the one from sdist, otherwise I’m not comparing them apple to apple. It will be a ‘recursive’
hash, and so is not expected to match the ‘flat’ hash from earlier, which is fine.

$ nix-prefetch-url --unpack file://<(cat from-hackage/hledger-1.22.2.tar.gz)
path is '/nix/store/17pf2cx2vrs9xicl6pkvvinlzcrxz1p4-63'

$ nix-prefetch-url --unpack file://<(cat from-sdist/hledger-1.22.2.tar.gz)
path is '/nix/store/17pf2cx2vrs9xicl6pkvvinlzcrxz1p4-63'

Now I know the tarball on hackage matches what was in the git repo for the version in question.

Then to ensure the sha256 is being checked (for my own peace of mind), I used an invalid sha256 in a fetchurl call

$ NIX_MIRRORS_hackage="" nix repl
Welcome to Nix 2.6.0. Type :? for help.

nix-repl> builtins.fetchurl {
  url = "mirror://hackage/hledger-1.22.2.tar.gz";
  sha256 = "0v6r3wwnsk5pdjr188nip3pjgn1jrn5pc5ajpcfy6had6b3v4dwm";
error: hash mismatch in file downloaded from '':
         specified: sha256:0v6r3wwnsk5pdjr188nip3pjgn1jrn5pc5ajpcfy6had6b3v4dwm
         got:       sha256:1g1v56fxgs7ya8yl22brwgrs49a50kd77k8ad8m8l5cnlnviqb3g

Nice. But to check that hackages-packages.nix benefits from the same check, I replaced in hackage-packages.nix (locally)

mkDerivation {
pname = "hledger";
version = "1.22.2";
   # sha256 = "1g1v56fxgs7ya8yl22brwgrs49a50kd77k8ad8m8l5cnlnviqb3g";
   sha256 = "1g1v56fxgs7ya8yl22brwgrs49a50kd77k8ad8m8l5cnlnviqb3h"; # <- notice the last character was changed to 'h'

And in order for the nix-daemon to fetch tarballs over HTTPS (HTTP packets being dropped and causing timeouts)

$ sudo systemctl edit nix-daemon.service
$ sudo systemctl daemon-reload
$ sudo systemctl restart nix-daemon.service
$ sudo systemctl show nix-daemon.service | grep Env

I then expected to get a hash error upon building the derivation where I injected an error in the hash:

(in nixpkgs-f8f124009497b3f9908f395d2533a990feee1de8)
$ nix repl .
Welcome to Nix 2.6.0. Type :? for help.

Loading '.'...
Added 14996 variables.
nix-repl> :b haskellPackages.hledger_1_22_2
error: hash mismatch in fixed-output derivation '/nix/store/iz354wx38sqq87vn7ia8m8zm7jm0liwk-hledger-1.22.2.tar.gz.drv':
         specified: sha256-cCwct6WWFYoqagrNc9oERSWi8+N5CUE9Uv7o150pO7w=
            got:    sha256-bywct6WWFYoqagrNc9oERSWi8+N5CUE9Uv7o150pO7w=
error: 1 dependencies of derivation '/nix/store/k6si40n8drn94sba0c0cz5460s8vnb92-hledger-1.22.2.drv' failed to build
[0 built (1 failed)]

Nice. Notice that the hash mismatch error message is slightly different, referring to ‘fixed-output derivation’ compared to the earlier ‘file downloaded’ hash mismatch.

So now I can build older versions knowing that I understand how source tarballs are obtained and checked. And If I need to, I can produce my own tarballs (at last for haskell packages). And I can also activate HTTPS (if I so desire).

I found the fixed derivation hash mismatch error message in pkgs/common-updater/scripts/update-source-version so I now know where to look if I want to understand what’s going on. I won’t need to just yet.

Putting this lengthy reply in case it will help someone else someday.