How NixOS and reproducible builds could have detected the xz backdoor for the benefit of all

Links
1

Hello everyone,
Do you remember the xz supply chain security that happened last year and nearly compromised half of the world?

I would like to share a blog post I published recently, in which I explain how NixOS could have detected the xz backdoor thanks to reproducible builds. Don’t hesitate to read it and tell me what you’re thinking about the idea.

35 Likes
2

As a sidenote, fetchFromGitHub does not use unzip.

$ nix why-depends --derivation nixpkgs#systemd.src nixpkgs#unzip
'flake:nixpkgs#systemd.src' does not depend on 'flake:nixpkgs#unzip'

It uses .tar.gz archives:

$ nix derivation show nixpkgs#systemd.src | jq .[].env.urls
"https://github.com/systemd/systemd/archive/v257.3.tar.gz"

And tar and gzip are both tools within stdenv.

As another sidenote, the more we use fetchFromGitHub, the more annoyed I am that it unpacks this archive. 99% of the time, there’s no reason to store the uncompressed, unpacked files. It takes a lot more space, and it’s much harder on the FS to enumerate or eventually delete a tree of small files than a single large one. I really think there ought to be an unpack toggle parameter, and although ideally it would default to false, for compatibility’s sake we’d have to just add it to all instances in nixpkgs.

8 Likes
3

Either this is a recent change, either I completely hallucinated that (which is also possible). I will investigate.

2 Likes
4

After investigating it a little bit, I think I hallucinated it and was confused by the fact that github releases are .zip if you download it from the web interface. In any case, I realize this argument was not very well done. I think the reason we build xz from the maintainer tarball has more to do with the configure script being already present there than the format of the archive. I’ll rework that part of the post.

6 Likes
5

Contesting opinion: that’s a Nix issue and, on the contrary, we should replace all occurrences of fetchTarball&c (NB: ElvishJerricco’s comment about names) with their unpacking variants because:

  • It’s better for deduplication (castore it)
  • It’s more useful to attest hashes of the content than of the specific containers
1 Like
6

for the record, fetchTarball specifically does also unpack.

I wonder if this is the actual reason that we do this. Maybe GitHub tarball archives aren’t stable, even if their contents are.

4 Likes
7

GitHub only guarantees unpacked source hash stability, so we can’t really rely on fetching archives without unpacking.

Also, fetchFromGitHub wants to be compatible with fetchgit, so both have to have a very similar output (i.e. unpacked)

4 Likes
8

Really cool read overall, and I learned something new that I actually wanted to ask more about.

In your article you mention how Nixpkgs is looking to implement something for bootstrapping the stdenv, inspired by Guix. Is there anywhere I could find more info on the Nixpkgs side of things, such as a GH issue or Discourse post?

9

I’d recommend starting by reading and following links for this PR.

3 Likes