Http links in nixpkgs

Development
1

Executing grep “http://” -r in nixpkgs yields 10k+ results. While some of those links are there because some websites just don’t support https, a bunch of them do, so there may be an interest to changing them back to https. We are checking for checksums in nixpkgs after all, but more advanced MITM attacks could still happen. Wanted to get others’ opinions before actually filing an issue in nixpkgs.

5 Likes
2

One foreseeable problem is that some of these, while if converted to https might work, certificates might expire in the future. At the scale of thousands… how often might that occur?

2 Likes
3

There’s no reason to not use TLS in nixpkgs if the upstream links support it, especially for src urls. Though the hosts should be verified to actually support it rather than a naive search-and-replace.

4

Doesn’t the sandbox builder not have internet access and verify hashes during build? I’m not sure what else TLS would be useful for in this context.

5

Are we seriously arguing against TLS in 2026?

4 Likes
6

I’m against security changes that don’t have a threat model yes.

7

If the certificates are expiring (e.g. the host isn’t using acme), what’s to prevent the domain name reservation from expiring also? Or the hosting? The fact is that websites don’t guarantee posterity in general. The internet is volatile. Isn’t this one of the problems that binary caching (or specifically the binary cache) is meant to address?

3 Likes
8

Ok, read the nixpkgs manual then :slight_smile:

Bikeshedding uncontroversial, objective net improvements is exactly why nixpkgs is so miserable to interact with for contirbutors, btw.

3 Likes
9

One threat model is that we’re exceptionally easy to fingerprint.

5 Likes
10

Hashes need to be updated (e.g. with nix-prefetch-url or with setting the hash to the empty string). I like that part to happen over TLS to avoid funny attacks

3 Likes
11

See if there’s real reason for it, then yeah we should switch to TLS. Even if it may break things.

12

I struggle to see why this proposal sparks an argument, TLS has been a thing for decades and we’ve pushed really hard for it to be used everywhere, especially when pulling software blobs.

Regarding the cert expiry issue, I don’t think this is such a big issue, if you have certbot running or a certificate with a long enough expiry date. Besides, I don’t think that we should adopt a lower security standard for thousands of packages if only a small amount of them are actually not offering HTTPS. This definitely should be done somewhat intelligently as to not break packages.

2 Likes
13

MITM attacks don’t exist, they can’t hurt you

Jokes aside there is real threats to not using TLS, especially if we don’t have a reason not to.

1 Like
14

I think I will work on a way to automate this, sounds like an interesting challenge.

1 Like
15

For anyone curious about the current state of affairs, I threw together a command which directly detects through evaluation packages that are using http links:

export NIXPKGS_ALLOW_UNFREE=1
nix run github:NixOS/nix-eval-jobs -- \
  --flake github:NixOS/nixpkgs#legacyPackages.x86_64-linux \
  --no-instantiate \
  --impure \
  --workers 8 \
  --apply 'value: { isHttp =
    value ? src &&
    value.src ? url &&
    (builtins.substring 0 5 value.src.url) == "http:";
  }' | grep isHttp\":true

Yields 924 results. Granted this is only for packages which show up under legacyPackages.x86_64-linux, but that’s going to be the vast majority of them, and is a good starting point.

16

Don’t forget about things like this. (There are many other similar cases to consider, I fear; patches were just the first thing that occurred to me as not to be caught by something like this.)

17

Tweaking the command to also evaluate patches yields 931 results, thanks @rhendric ! Let me know if you think of any other edge cases, meanwhile I think I’ll just spend some time grepping for “http://” to see what happened to those other 9k results from the OP.

nix run github:NixOS/nix-eval-jobs -- \                 
  --flake .#legacyPackages.x86_64-linux \
  --no-instantiate \
  --impure \
  --workers 8 \
  --apply "value: {
    position = value.meta.position or \"unknown-position\";
    isHttp = (
      value ? src &&
      value.src ? url &&
      (builtins.substring 0 5 value.src.url) == \"http:\"
    ) || (
      value ? patches && (
        builtins.foldl'
          (acc: x: acc || ((x ? url) && (builtins.substring 0 5 x.url) == \"http:\"))
          false
          value.patches
      )
    );
  }"
18

OK, if I just limit my search to pkgs/**/*.nix, then I get 3187 results. 1520 of those are homepage links, 193 are comments, 330 are old unused thunderbird archive.mozilla.org urls, 40 are from pkgs/build-support/fetchurl/mirrors.nix, which leaves a difference of 173, which from a very cursory glance seem to be all sorts of different things not related to source URLs, but I could easily be overlooking things.

19

So far good progress! I filtered the ones with http sources into a file called only-http.txt, removed the ones that had evaluation errors, and am running the following script to generate commits. I’ll update this comment as I make tweaks to the script. I’m basing all this work off of nixos-unstable to take advantage of the cache.

#! /usr/bin/env bash

set -e

trap exit SIGINT

while read -r line; do
    attrPath=$(echo "$line" | jq -r '.attrPath | join(".")')
    position=$(echo "$line" | jq -r .extraValue.position)
    position=${position%:*}
    position=${position#/nix/store/*-source/}
    echo
    echo "PROCESSING $attrPath in $position"
    parent=$(dirname "$position")
    # out=$(nix build .#${attrPath}.src --print-out-paths --no-link)
    # nix store delete $out
    for f in $(find "$parent" -type f); do
        sed -i 's/url = "http:/url = "https:/g' "$f"
    done
    if ! git diff --exit-code; then
        if nix build .#${attrPath}.src --rebuild || nix build .#${attrPath}.src --option substitute false; then
            git add "$parent"
            git commit -s -m "$attrPath: use https for sources"
        else
            git stash
        fi
    fi
done < only-http.txt

Actually, since forcing a rebuild of patches is difficult, I’m going to focus on just sources for now, and will circle back to the handful of http patches.

Also, any suggestions for parallelizing this would also be most welcome! Some of these have to wait to timeout 4 times, which really hampers progress.

2 Likes
20

I wasn’t intending on starting an argument. I was genuinely asking what threat models exist that TLS would solve that the sandbox doesn’t.

1 Like