Nixpkgs supply chain security project

Announcements
1

Hi everyone,

a couple of months ago, back in June 2023, multiple groups of Nix ecosystem contributors prepared project proposals for the Sovereign Tech Fund “Contribute Back Challenges”.

Today I’m glad to announce that the Nix ecosystem supply chain security project will be supported by the Sovereign Tech Fund with an investment of 226 000 Euro!

The focus of this project is on reducing our reliance on foreign binaries to compile Nixpkgs from scratch, ensuring we are are indeed running the code we compiled by leveraging existing security components in NixOS, and putting in place mechanisms that allow us to deliver the most up-to-date, secure software whenever it is available in a way that can be sustained given our maintainer capacities.

Specifically, by the end of the year this will provide the Nix community with three major improvements to the ecosystem’s security story:

  1. A state-of-the-art vulnerability tracker for Nixpkgs
  2. TPM2-based UEFI Secure Boot for NixOS
  3. A full-source bootstrap chain for the Standard Environment (stdenv) in Nixpkgs

Check the submitted project description for details on the planned deliverables.

All of these three capabilities combined in one software distribution will be quite a unique feature in the open source world.
Nix is not only particularly suitable to support that with declarative, immutable configurations making highly intricate setups tractable.
The Nix ecosystem also needs them very much due to its immense size, growing popularity, and the resulting attack surface.

I’m quite excited about this happening, and proud to be part of it.
This effort will, with time, greatly improve the security situation for most Nix and NixOS users by default.
And it will also very likely further boost adoption of Nix and NixOS in security-sensitive software development communities.
It’s already an amazing showcase of community collaboration, and a perfect opportunity to show yet another time what’s possible with Nix.

@raitobezarius will work on the boot chain security part, together with @nikstur and @lheckemann. @raitobezarius is a Nixpkgs security contributor and reviewer of security fixes, the and was release manager for NixOS 23.05. @nikstur is one of the maintainers of lanzaboote, and @lheckemann is a prolific long-term contributor to Nixpkgs and NixOS, having managed the releases of NixOS 19.03 and 19.09 as well as being a member of the security team and the RFC Steering Committee.
@thubrecht will set up the web service for the vulnerability tracker and collaborate with the Nixpkgs security team to address their requirements.
The bootstrapping work is led by @emilytrau, who kickstarted the minimal bootstrap project and is involved in the community that delivered the Guix full-source bootstrap.
I will support the team with administrative tasks, external communication, and documentation reviews.

Thank you very much to everyone who made this possible!
Special thanks to the NixOS Foundation board (@edolstra, @domenkozar, @regnat, @ron, @zimbatm) for both financial and moral support with the application process.
Extra special thanks to @proofconstruction for sitting through the arduous last checks of the project descriptions.

The Sovereign Tech Fund (STF) invests in many other open source projects you may be interested in. Check out their announcements for this year’s funding round. STF supports the development, improvement, and maintenance of open digital infrastructure in the public interest. Its goal to strengthen the open source ecosystem sustainably, focusing on security, resilience, technological diversity, and the people behind the code. STF is funded by the German Federal Ministry of Economics and Climate Action (BMWK) and hosted at and supported by the German Federal Agency for Disruptive Innovation GmbH (SPRIND).

Stay tuned for updates, and get in touch if you have any questions.

81 Likes
2

Those are great news! All of those things are sorely needed to make Nix and NixOS a solid choice for businesses and private users alike.

So cool to see that capable contributors will get paid to do good work, I hope that helps with their general workload as well.

The STF announced it would take longer to review the applications, but I didn’t expect them to take two months longer than initially planned.

2 Likes
3

This is really fantastic; thanks to all of those who are helping to achieve these goals, it’s much appreciated.

1 Like
4

I am looking forward to the minimal bootstrap efforts to hopefully one day get rid of the bootstrap archives (at least for the common platforms)! It seems there are still some challenges to overcome there. Is there an issue tracking this, maybe?

The vulnerability tracking sounds interesting, I’d like to see how that turns out.

2 Likes
5

Thanks for the support! Progress on minimal-bootstrap is being tracked in this PR https://github.com/NixOS/nixpkgs/pull/227914

13 Likes
6

Just a huge THANK YOU shout out to the folks who did all the incredible work that made this happen! <3

3 Likes
7

Here’s the tracking issue for NixOS boot security: Tracking issue: Boot security in NixOS · Issue #265640 · NixOS/nixpkgs · GitHub

3 Likes
8

About a month left to land the promised deliverables. Here’s a status report.

tl;dr We’re on track.

Since the initial announcement, the working group has grown as the specification became more refined, with every milestone having at least one individual responsible.

NixOS Boot Security

  • @lheckemann got an ISO to build that can be booted with Secure Boot enabled via shim. It currently works by signing the kernel directly. It doesn’t verify the initramfs or store image, so the next step is to build a Unified Kernel Image to get the initrd verified, and do a checksum of the store image to get all the desired security properties. This is not ideal, especially performance-wise, but will unblock kicking off the shim-review process to obtain a signature that allows booting on machines that only ship with Microsoft keys.
  • @ElvishJerricco explored multiple options to verify the store closure
    • The most secure and attainable solution for now is using nix-store --verify on regular systems, despite a significant performance penalty, and dm-crypt on appliances.
    • At the end of the year there will be a an outline for what an optimal solution would loook like: something very much along the lines of APFS Signed System Volume, with ZFS being closest to being able to implement it, despite being far from ideal in many other respects.
  • @nikstur opened a tracking issue: Perlless Activation - Tracking Issue · Issue #267982 · NixOS/nixpkgs · GitHub
    • Booting NixOS without running the Perl interpreter is almost done
    • At the end of the year we’ll have a clear implementation strategy to also get rid of Bash in a possible next round of work
  • @RaitoBezarius
  • @Dmills27 started working on documenting the complete setup for end users

Vulnerability tracker

  • Due to a constellation of unfortunate situations, we had to quickly re-plan some milestones. Special thanks to Ryan Trinkle and Ali Abrar from Obsidian Systems for jumping in to support us with developer time and project management expertise on short notice, with @cidkid being part of the group now.
  • Progress is tracked here: Issues · Nix-Security-WG/nix-security-tracker · GitHub
  • @thubrecht implemented the web service infrastructure and CVE ingestion, and is working on providing APIs to implement clients against
  • @raboof and @cidkid, after an extensive survey of prior art, are developing a tool to match local store path closures and metadata against advisories, which will use the web APIs to improve accuracy.
  • @mightyiam, @jfly, and @modprog work on extending and integrating label-tracker, a tool to track PRs/issues in nixpkgs according to their tags, generously supported with input from @pennae

Nixpkgs full-source bootstrap chain

  • @emilytrau is tracking progress with PRs referencing https://github.com/NixOS/nixpkgs/pull/227914
    • Implementation is in the home stretch, with GCC 8 compiling with musl, bootstrapped off the latest stag0-posix release 1.6.0
    • There was lots of volunteer support in the form of reviews, thanks @picnoir, @artturin, @Ericson2314 and everyone else involved!
  • @alejandrosame is writing in-code and maintainer-oriented documentation, since a major contribution to security by this sub-project is being able to understanding how and why things work

Phase 2

Overall, the project is on a path to fulfill the envisioned requirements in terms of security, but due to time limits has to strike many compromises between implementation complexity and performance. One important byproduct are findings that inform possible next implementation steps.

The second application phase is coming up, where we can continue to build upon what’s currently being done and incorporate those findings. We invite everyone available to work on security in the first quarter of 2024 to participate in planning. The deadline to hand in a proposal to the STF is Wednesday 2023-12-13.

14 Likes