We built a hardware-attested Nix build! Details at Garnix Blog: Hardware-Attested Nix Builds
11 Likes
This is a great achievement, congratulations.
I’d be really excited about seeing this done on top of content-addressing derivations with laut as the signature format, so that the verification can be done in a way where it links up all of the individual derivations. Even if the Merkle-ness of the hashes that you have for your inputs properly include the content-hash-based identity of the whole runtime closure of the build step, that only tells you something based on assumptions you need to make about how those inputs came to be originally (why is any one particular dependency trusted and in the closure?), and I think linking up all of that evidence during verification would be the next step.
Of course your customers can already trust you with how your system is set up in terms of which dependencies are trusted, but going further like that could enable distributing trust and migrating to stronger guarantees over time. I’d be happy to talk about that.
3 Likes
Yeah, You have thought much more deeply about the signature format, and it’d be great to talk about it at some point!
only tells you something based on assumptions you need to make about how those inputs came to be originally (why is any one particular dependency trusted and in the closure?),
Not sure I understood this. When you say “why is it trusted”, do you mean why is it believed that that input with that output hash, is a correct version of the input (i.e., integrity), or why do we trust the input’s input at all? Or do you mean something else entirely.
One thing that ended up being somewhat different in our approach than what I think you have in mind in your paper is that for us the virtualized TEE is a substitute for the Nix sandbox (not a container for the entire eval-to-build process, or even just for the remote builder). So it can’t attest anything about how we ended up with a particular set of inputs at all, since it starts its life with all those decisions already having been made. I don’t think that’s very significant, but it might change a bit how you segment attestations. (This could be a nice thing anyhow, since at the build stage we can’t really avoid using a virtualization-based TEE (but can avoid e.g. network), but at other stages we can (but e.g. can’t avoid network), so it feels like cutting things this way could lead to a pretty precise picture of what steps depend on what.)
What I described in the paper is independent of this decision I think, but I agree with what you did, because it saves you from the security of the sandbox being one of your trust assumptions.
On some level you can think of accessing an input-addressed store path as a kind of dependency resolution, because the computation of the store path is completely independent of who you trust, but the contents that end up there, and in turn factor into your build is very dependent on who you trust.
So recording the hashes of these dependencies in a signature or attestation is part of the solution, but not the whole solution. I suspect that the answer to
is that it was built and verified in the same way, but maybe you also trust inputs signed by nixpkgs, etc. This is a matter of how Nix, or your deployment of it or your product specifically, is configured.
I would argue that in a well designed system, these configuration decisions should not play any role at all that isn’t clearly visible on the verification in some way. To accomplish that, you have to make the verification procedure stricter.
Instead of just verifying the attestation of the final build step, you would obtain an attestation for every single build step, and then carefully look at all of the input hashes, to make sure they all line up, so that they are actually talking about the same thing.
One way to describe this process, and what it looks like on the implementation side in laut, is (re-)doing the dependency resolution on the verification side.
With content-addressing derivations in Nix, this process ends up looking much more naturally like dependency resolution, because instead of worrying which contents were actually at a particular store path and reverse engineering that, the process of going from a derivation to the basic derivation (I’d really like to call these unresolved and resolved derivation going forward) where all the inputDrvs replaced with inputSrcs, yields an entirely different basic derivation based on the different content-addressed store path of a build input, and therefore based on who the builder trusts. On the consumer side, if who you trust is the same, that turns into a cache hit, if who you trust is different, that turns into a cache miss.
The specific data structure in the cache that this lookup happens on is called a build trace, it used to be called the realisation of a content-addressing derivation, but that term was quite overloaded, so we’re trying to change it.
I strongly believe that making remote attestation approachable is important, but I also believe that content-addressed derivations are a the right technical basis for doing that correctly,
since some technical basis that makes Nix take some content-hash based identity into account with these decisions is for sure necessary. This necessity also applies to any quorums or other schemes for aggregating evidence that we come up with.
@Ericson2314 and me are working on an RFC to stabilize CA derivations, which will have to specify a signature format for these build traces, and providing a solid basis for remote attestation and quorums will be one of the goals there.
I think what I am saying here might sound quite alien and unnecessary to some, because everywhere else in software it’s a law of nature that you have to trust who the people you trust are trusting, so it is genuinely hard to start questioning that in places. We’re not violating that law, but we’re starting to push back on it in practical terms, because it gives us useful properties. It makes it possible to change who you trust over time, migrate to stronger attestation mechanisms or throw a specific party out of your quorum, with dependency resolution doing the necessary work so you don’t throw out more than you have to. As far as I know nobody else is doing that.
2 Likes
I think this is (in terms of functionality) what I was trying to get at with having the builder attestation data include (input hash, hash of all built outputs, output hash). Metadata would need to be accessible out of band for translating e.g. the hash of all built outputs into the full map of (input → outputs). But once you have that, the consumer can, when deciding whether to trust the final artifact, query all its caches for that input with that matching output, and only accept the final artifact attestation if it finds such an input and accepts the signature.
I said “in terms of functionality” above because while (if I understand correctly) this achieves the same security goals, I completely agree that the CA approach is much more elegant (very elegant, in fact; and it also simplifies some things we are doing).
1 Like