CVE-2024-3094: Malicious code in xz 5.6.0 and 5.6.1 tarballs

[szlend]

Really wish there was a better way of rolling out these sort of security updates.

I assume we can’t just bake replaceDependency into the default evaluation of nixpkgs can we? Would be interested to hear what sort of problems that would cause, having never tried it before.

I’m not sure how scalable Hydra is or what sort of budget the nix organization operates with, but a bruteforce approach could also be to just throw more temporary hardware at the problem.

Another approach could be to define a subset of core security sensitive packages and only update xz for those as a first pass. (e.g. pkgs.openssh would depend on pkgs.securitySensitive.xz that can be updated independently of pkgs.xz).

[randomnixuser]

Can someone bake a command to run vs the nix store to check which packages use the xz version 5.6.x? I was not able to do it in a nice why with --why-depends

[norpol]

From your linked posting:

[…] releasing an update with the backdoor code removed (xz 5.6.1-2) […]

They don’t have a 5.6.2 version, they’ve patched 5.6.1 to exclude some of the exploit code.

10 days for a high security risk

At the moment there doesn’t seem to be a security risk on NixOS (similar to what OpenMandriva pointed out). The exploit was included during build-time. If you want to be extra safe you could move away from -unstable to 2311, which in general includes security fixes quicker than unstable (and doesn’t contain this CVE in particular).

[norpol]

@szlend

Another approach could be to define a subset of core security sensitive packages and only update xz for those as a first pass. (e.g. pkgs.openssh would depend on pkgs.securitySensitive.xz that can be updated independently of pkgs.xz).

That sounds a good solution, maybe we should open a separate thread where we discuss faster ways of pulling core package security updates into nixpkgs-unstable?

Note (for others, since you are talking hypothetically): openssh does not depend on pkgs.xz or systemd, you can check this yourself by looking at the source, or running ldd "$(which sshd)" | grep liblzma or nix derivation show "$(which sshd)" | grep -- -xz.

[norpol]

You can use just nix run nixpkgs#nix-tree if you want your check your current system, in there you can search for “xz-5.6.1” and on the bottom it will show “Immediate Parents”, which was also helpful to know when web searching for cli alternatives.

Alternatively you can use something like this:

nix-store --query --referrers /nix/store/yyqzw7xvsrn3h2zrvincbs1b291yzx8c-xz-5.6.1
nix-store --query --referrers "$(which xz)"

for my desktop system the result looks a like this (removed the full paths):

• altair-6.3.1-usr-target
• boost-1.79.0
• boost-1.81.0
• elfutils-0.190
• ffmpeg-4.4.4-lib
• ffmpeg-6.1.1-lib
• ffmpeg-full-6.1.1-lib
• ffmpeg-headless-6.1.1-lib
• fwupd-1.9.15
• gnome-disk-utility-45.1
• imagemagick-7.1.1-29
• imlib2-1.12.2
• kmod-31
• kmod-31-dev
• kmod-31-lib
• libarchive-3.7.2
• libarchive-3.7.2-lib
• libtiff-4.6.0
• libtiff-4.6.0-bin
• libunwind-1.8.1
• libxmlb-0.3.15-lib
• ostree-2023.2
• python3-3.11.8
• squashfs-4.6.1
• systemd-255.2
• telegram-desktop-4.15.2
• tor-0.4.8.10
• xz-5.6.1
• xz-5.6.1-bin
• xz-5.6.1-dev

[pancho]

I’ve gone old-school, and I’ve seen that even my emacs is affected.
Run this, redirect its output to a file, and filter/inspect as needed:

find /nix/store -type d -name .links -prune -o -type f -executable -print | xargs -n 1000 ldd 2>&1 | awk '
/:$/ {
    current_binary = $0
}

/liblzma\.so\.5/ {
    print current_binary $0
}
'

[t0m]

Thanks to everyone helping with fixing and with putting together information about this in Nix!

It would definitely be nice after this to think about making some easy to use tooling for potentially urgent situations like this. Some things I’d could imagine useful as well (maybe these already exist but I don’t know about them):

• A way to list the gcroots that would have to be removed for a store path to be gc-able.
• A way to prevent a package version from ending up in the local nix store, e.g. a blocklist on the nix.conf level to match glob *xz-5.6*.

Regarding the faster patching, I know that Guix has a mechanism they call Grafts. Does anyone know if there is something that Nix could adopt there to improve this or is that comparable to the NixOS replaceRuntimeDependencies solution?

[Sandro]

@szlend

I assume we can’t just bake replaceDependency into the default evaluation of nixpkgs can we?

No, Hydra doesn’t support it. Last time I tried it, the evaluator build everything and the evaluation log wouldn’t load anymore, making the projectset page a DOS for every browser visiting it. Besides that it is also impure.

[chrism]

This may be useful… should print out all referrers to xz-5.6.1

nix-store --query --referrers $(ls /nix/store|grep 'xz-5.6.1$'|xargs realpath|tr '\n' ' ')

(likely most useful after a sudo nix-collect-garbage -d)

[PerfMonk]

Is anybody able to confirm that nixos xz 5.6.1 is not infected ?

This would be really a reassuring thing to know that we are not at risk since the libzma library is a dependency of many packages.

[Roger]

I’ve created a patched version of xz and the resulting liblzma.so is almost identical to the one from nixpkgs, I guess that confirms that the backdoor is not even built in nix. (I’ve the tar in nix’s cache from before the repo was down).

--- pkgs/tools/compression/xz/default.nix	2024-03-30 13:12:48.754814282 +0000
+++ pkgs/tools/compression/xz-source/default.nix	2024-03-30 15:49:06.109915294 +0000
@@ -2,6 +2,7 @@
 , enableStatic ? stdenv.hostPlatform.isStatic
 , writeScript
 , testers
+, autoreconfHook
 }:
 
 # Note: this package is used for bootstrapping fetchurl, and thus
@@ -14,13 +15,15 @@
   version = "5.6.1";
 
   src = fetchurl {
-    url = with finalAttrs; "https://github.com/tukaani-project/xz/releases/download/v${version}/xz-${version}.tar.bz2";
-    hash = "sha256-0wBCJkmgEksRIWML5VnIkM7t8yZn1wZLgSiTMWbCF8g=";
+    url = with finalAttrs; "https://github.com/tukaani-project/xz/archive/refs/tags/v5.6.1.tar.gz";
+    # url = with finalAttrs; "https://github.com/tukaani-project/xz/releases/download/v${version}/xz-${version}.tar.bz2";
+    hash = "sha256-I3KE+uQOX46ZCPCpd+fQuaXHwcEKQbjm7Q+0DpMEZ8g=";
   };
 
   strictDeps = true;
   outputs = [ "bin" "dev" "out" "man" "doc" ];
 
+  nativeBuildInputs = [ autoreconfHook ];
   configureFlags = lib.optional enableStatic "--disable-shared";
 
   enableParallelBuilding = true;

[midchildan]

Just to clarify, since the currently known backdoor is inserted to the binary by the configure script, is it also safe to assume that Nix users on Darwin, Debian, RedHat, and others are also not affected?

[norpol]

From the original report, the payload was only included when this if case triggered during build. if test -f "$srcdir/debian/rules" || test "x$RPM_ARCH" = "xx86_64";then. As Nix builds are pure, aka won’t contain debian/rules or the variable RPM_ARCH during build, there shouldn’t be any official nix generated artifact that will contain the payload built in.

[hexa]

From what we know at this time, our 5.6.1 package is not affected by CVE-2024-3094. The top post says as much.

[sleepy]

Someone has put together an interesting read with general information known so far about the timeline of events going back to 2021:

Everything I know about the XZ backdoor

I get the impression that all of this was meticulously planned from the start.

[ada7urhb]

So “and liblzma remains unpatched.” actually means

“NixOS Unstable currently ships with xz 5.6.1 and is unaffected, as the malicious code path exits early, and therefore liblzma remains unpatched.”

Just so I fully understand the knowledge to date not wanting to bring multiple servers back online using Nix unstable if they are at risk.

[dpc]

@hexa

This is currently at the head of the queue, but since xz causes a stdenv rebuild on all four supported platforms, we are talking about a 220,000 packages in the queue that need to be built.

Is there any kind of ordering there?

What I’m thinking is: Sure, it takes a lot of time to build everything, but probably 10% of these are a core that all servers use, then some architectures are more popular than others, etc.

It should be possible to opt-in into some very-early-unstable-head-channel, that guarantees only a core subset built (toolchains and core utilities, possibly only for x86_64), get 90% of the build from the binary cache, then every user and organization can build whatever they are using outside of that core themselves.

Then I’m curious - is throwing more money at the problem scaling well here? As in - in own self interest, if NixOS users (e.g. organizations) funded more build machines, would things go faster, or are there some architectural roadblocks that might make it not as simple?

[redshift]

Others have now found evidence of malicious commits other than the build change that can impact SSH login. See: Yellow Flag: "The original xz maintainer started fixing the iss…" - Infosec Exchange

TL;DR: The commit git.tukaani.org - xz.git/commitdiff hid a . before the definition of a sandbox function, disabling Linux landlock. The long-time maintainer of xz reverted it today.

To me, this means that the first vulnerability that’s become so public was not their sole goal, and even if the build-time issue doesn’t apply, others may. That commit would be in the 5.6 and 5.6.1 releases as well.

[osmano807]

I would like to know if NixOS was warned during the embargo or if it knew at the public release, couldn’t find information about this.

I’m not too inclined in fully trusting the general assumption that it’s safe because we don’t use deb or rpm compilation.

Is there any guidance on applying replaceDependency to the entire store? As cited in Feature request: Replace package without rebuilding world · Issue #132749 · NixOS/nixpkgs · GitHub, should we focus on Content-addressed Nix?

[me0]

Hello NixOS Community !
A source based package and still has a backdoor due to building from a tarball instead of the git repo, so what about all the pre-built binaries in nixpkgs ?.