Proposition: I think we need DJ to stay pure?

Development Nixpkgs Architecture
1

Hi :blush:

The problem to solve:

We need randomness (for example, for key generation), but since it’s fundamentally impure, we handle these tasks “manually”. These impure operations can become tedious and a source of problems, much like the many manual operations in other distributions. It is fundamentally impossible to purely describe a machine that partly needs to be the result of randomness.

A clean generic solution:

The description of machines also includes the description of impure elements (the data to be produced containing randomness) — that’s the “DJ” (Dirty-Job) part. DJ does not produce randomness; it purely produces code that handles this randomness (which is why everything remains pure). The build includes an extra step: before the usual Nix build, the code resulting from DJ is produced and executed. It performs the impure actions if needed, then the build is carried out.

I envision two modes:

• Clean mode:
◇ Two repos
:black_small_square: One without randomness (no public or private keys or other files originating from randomness or the “outside”).
:black_small_square: One like the first, but also containing the result of DJ execution.
◇ The “DJ” phase consists of producing the dirty repo (the 2nd) from the clean repo.
◇ The classic build phase.
• In-place mode: Same thing, but everything in the same repo.

Drawbacks

One extra step.

Advantages

• Everything can be automated; we only have to produce Nix code.
• No risk of forgetting to generate keys, to renew them, no secrets.nix to manage — we can replace agenix-rekey with a cleaner solution.
• Automated collection and management of public keys for different machines.
• Permanence management: if a key is no longer needed, it isn’t automatically deleted; we can configure how long before the key is destroyed.
• Distribution of DJ modules: administrators no longer need to know how to generate keys; they just need to know how to configure them. This allow to always use the state of the art practices just by updating DJ modules.
• Distribution (in clean mode) of sources with or without the randomness parts, depending on the use case (with them internally, without them for sharing knowledge).

• No need to produce a manual of the “outer part of the job” because everything is in the nix files.
• Later, possible integration into Nix to make the DJ process natural.
• DJ is not limited to keys but to any manual part of the repo. For example, generating automatic documentation, using an additional code analyzer.

Example:

config.dj."${name}".httpd.key.enable=true
...
sslServerKey=if devMode then config.dj."${name}".httpd.key.path else <ECMA things>;

That’s it, the rest (key pair creation, rekeying, determining which machines use these keys) is handled by DJ.

--

I’m an enthusiastic beginner on NixOS, unfortunately completely swamped with other things, but maybe this idea could be beneficial and might interest some developers.

-----BEGIN SSH SIGNATURE-----
U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgE3auYztV18qug7NHVal89ZH0U5
jnKZKonRNR5Y5cndkAAAAEZmlsZQAAAAAAAAAGc2hhNTEyAAAAUwAAAAtzc2gtZWQyNTUx
OQAAAEDvw8IDxKvweK+FUKmFZ8aCCpywvgabwQ+OYFjafDekKkpcM6iXeEzl3vNGNsif7R
TXF6cbpmFs7ys74BVOXGkG
-----END SSH SIGNATURE-----
2

Nix is already completely capable of generating random files at build time (and yes, this means nix builds are not in fact 100% reproducible without some effort), and IFD means that you can import them. This is already possible.

You should not however do so, because /nix/store must by definition be world-readable and hence any secrets nix produces would be visible to any process that can read files on your host. This is why agenix/sops-nix & co. exist.

Personally I think agenix/sops-nix & co. are the wrong solution, though. It’s pretty easy to set up a systemd service that runs After=systemd-random-seed.service and generates your secrets at boot time, which is anyway better all around.

6 Likes
3

I’m curious what part of the build process is lacking here? Where did you encounter something that needed to be truly random but wasn’t?

I agree with @TLATER, nix/os should not be responsible for randomness. If every part of your config is declarative and pure everyone can reproduce it.

Basically the problem boils down to how we encode state in NixOS. If you ask me, just dont. NixOS should define what state is required (ex the /etc/host_ssh keypair), and not how it got there. Then nixos can be seen as a function over arguments: hostname, keypair, etc. This way we quarantine the impurities.

1 Like
4

Why the criterion of a pure function is not sufficient from the administrator’s point of view:

Producing a machine with NixOS consists of describing a pure function that only has elements from the repo as parameters.
But what interests the system administrator is not quite that: it is to work only on an abstract description, do rebuilds, and that’s it. And the difference is significant: as a pure function, one could take the identity function and ask to have a disk image of the system to copy in the repo. It would be pointless, would leave all the work to the administrator, but it would still be a pure function.

Why currently part of their work cannot be represented by .nix files: because one must generate random data (e.g., keys). So of course, as I explain above, one could consider that it is not NixOS’s job to do this. But that would mean missing out on a suitable tool. Indeed, agenix-rekey meets partialy this need, but it is, in my opinion, a solution that is not global and is rather “tricky.”

5

You can’t generate a secret at boot time if you need to use a other secrets (to certify). And even if you can you need to import the public key in your repo, so the problem is not resolved.

6

It depends: there are a number of answers to this question, but I think the tension at the bottom is that there is no particular reason to bind a secret / root of trust to a particular nixos generation. It seems much more sensible to have the machine bootstrap it’s own root of trust and for the configuration to reference it.

i.e. for baremetal use random host keys, which can be generated on boot and pushed to inventory management (saltstack does this for agents). Or random hostkeys that get pushed to boxes (but outside of nixos) i.e. colmena and nixops both have this notion of secrets outside of hosts. Or ideally device bound secrets which can be pulled or polled off the machine as part of installation or bring up.

With public / private clouds you can establish roots of trust from secrets injected into the VM. In this way you aren’t trying to bind trust with your nixos configuration and I think you’ll be happier for it. Minor anecdote: we happily (on aws) use iam credentials to seed everything outside of nix (currently aws ssm parameter store, future openboa), but inject secret locations into nixos from terraform (which proves the object/secret location exists when building the launch template). So we don’t need nixos-rebuild switch for instance to rotate a secret.

If you don’t care about any of this because you are managing like 2 boxes or something then I guess agenix/sops + git for mostly static secrets is fine, but it’s not the route I would take. I have never come across a strong reason to bind the value of a secret to a particular nix generation, only it’s location.

1 Like
7

Then you also cannot generate such at build time without compromising the certifying secrets.

These kinds of secrets are better encoded in runtime secret management schemes, like vault, or password managers for desktop use cases.

Systemd creds support reading from sockets for this use case. I’d recommend looking into systemd-creds if you want to do declarative secret management in general, it’s an excellent interface for this (though sadly given how NixOS modules are often designed - partially because of sops-nix/agenix’ interface locking secrets to file paths - you often cannot rely on NixOS upstream if you want to use them).

Again, you absolutely can do this, it’s just horribly insecure. pkgs.runCommand "random" {} "openssl rand > $out" will generate random data and dump it into a store path at build time.

Without an encryption scheme of sorts you end up exposing this to all your host processes, however, and if you want to fix that you end up reinventing agenix.

Maybe an alternative that meets your use case in fact is an agenix clone that runs some scripts to generate the encrypted files you include in the nix store, but I still maintain that this is simply the wrong way to think about most secrets; they’re data, and should not be set at the configuration layer.

1 Like
8

You might find interesting https://github.com/NixOS/nixpkgs/pull/370444 and Vars: A framework for managing secrets and computed values | Clan for some prior work on using code to document secret generation schemes. At the bottom of the first PR is also a GSoC project with similar goals.

3 Likes
9

Thanks it’s very interesting and match the problems I have.