Right, that’s true. None of these so far have been mass rebuilds because servers / leaf packages. It’s also true that most CVEs come with a work around that doesn’t necessarily require a rebuild (disable this kernel module, disable this protocol, change this config option etc), a lot of our patch stack looks like that. They are also server builds with a dramatically smaller package set in general. For my desktop I just firewall the shit out of it, and if there was some exposed component I’d hope for an alternative (firefox → chrome, unbound → dnsmasq) that I could temporarily swap to.
Almost a month has passed since NLnet Labs released Unbound 1.25.1 on May 20.
As a NixOS user, I’m worried about the security implications of fixes like this sitting in staging for weeks, especially when private companies are starting to present themselves as more secure alternatives.
Is the main bottleneck here Hydra/build capacity? If so, could this be meaningfully mitigated with more compute resources? Not excluding other smarter approaches.
Everyone familiar with the staging timeline could have painted you a picture of the expected timeline. The fix will land in nixos-26.05 end of this week and in unstable probably around end of month.
The new queue-runner increases our scheduling capacity, which may allow us to then increase build capacity, and we are well on our way to test it out. Once it starts working we’ll add more build capacity and that will hopefully accelerate the time to deliver package updates via staging.
That being said, the staging cycle is here to collect high rebuild changes and repack them into fewer total rebuilds. In the current model there absolutely needs to be a cut-off point to protect the channels from being blocked for extended timeframes. Unbound is currently way beyond that with ~34.6k rebuilds.
6 Likes
The incentives are difficult here.
It’s tempting not to contribute to the shared tools/code (if not infra itself) and market that as your product’s advantage. On the other hand, lots of work on the shared nixos.org stuff is being “done for free” by various people; in a company you’d get paid for the same instead. So while pooling the human resources is surely more efficient overall, privatizing them may seem better for many individual companies.
1 Like
Removing unbound from gnutls is a good call, but it’s more about spreading the load than fixing the actual bottleneck—a critical CVE still has to wait in staging regardless. Seems like two separate problems, really.
1 Like
The latter problem is also discussed in the thread. It’s a consequence of nix’ decision to side with the static linking side of the static vs dynamic discussion. There isn’t much of a way around this without nearly a full redesign.
Skipping staging would not necessarily help, it’d just cause mass rebuilds that also take forever, and cause other CVEs in the pipeline not to be fixed. Increasing build capacity can help, and is being worked on, but almost certainly won’t get you to 1 day updates either.
@rnhmjoj and @bryango 's replies cover most of that side of the issue, including the potential solutions.
In the short (and most likely long) term, if this kind of thing is an issue for you, you should simply not be using NixOS. The only other option is the manual work to take on downstream builds for CVEs in packages you consider critical, like @bme. The -small channels can help.
2 Likes
Spot on. I’ve been noodling on a nix-a-like that uses GitHub - libabigail/libabigail: The ABI Generic Analysis and Instrumentation Library · GitHub to bundle up at build time what the ABI for c-libs, and design a store structure around that, so that you’d have a sound basis for automatic grafts. I’ll probably be in the ground before I find time to make it alpha quality 
But if anyone else wants to have a go…
2 Likes
I also ended up generating a list of overlays and patches for the main issues affecting my systems, similar to @bme ‘s approach.
Since these use a limited number of packages, building those was not a big issue.
This makes me wonder whether this approach could be extended further as a community maintained set of overlays and patches specifically for security issues. This would be aimed at those ready for some rebuild 
(and maybe a shared build cache for a small subset of affected packages only).