I deploy a server whose primary role is to be a backup server (zfs auto-snapshot are used to avoid deletion without root access). But since my ssh keys is stored and used on my main computer I’m worried that a malicious script may be able to corrupt my laptop user, e.g. to inject a malicious ssh to steal my keys and password before erasing all my data with root access on the server. A way to mitigate it is to enable 2FA with ssh so that connection for the root user also requires a one-time password stored on my phone. That’s better but still not perfect: if ssh is compromised on my laptop, the attacker could still theoretically run arbitrary commands during that session (ok I may be a bit paranoid…).
But I think that with nix we can do better, i.e. creating a server where the root account is never used: deployment could be made by pushing the config to a git server, and a web interface could be created, displaying the diff between the currently deployed version and the version to deploy, and this web interface could wait for the approval of multiple clients (e.g. phone + laptop) before deploying the change, possibly linked to a CI pipeline to follow errors etc.
Has anyone already implemented something like that?
You could pretty easily add an ExecStartPre to the autoupgrade unit that fetches a git repo (or straight-up use the flake feature), and host that git repo on a forge with 2FA. Put forgejo behind authelia or something.
That said, an even better and much simpler solution is to just get yourself a yubikey and use that for SSH. A security key completely prevents the types of attack you’re listing here - literally only stealing the key you physically carry with you, and knowing your pin (a second factor…), would give someone ssh access at that point.
That, or bugs in ssh or MITM attacks, of course, but you can’t avoid those with other systems either.
Autoupdate is a cool idea but sadly it is pull-based so I don’t want to wait e.g. 5mn between any deployment.
And sadly, none of these solutions (2FA on forgejo or even yubikeys) are secure if my laptop is corrupted. The reason is that the attacker can simply provide a malicious ssh on my machine (or for forgejo a malicious browser) that waits for a valid connection, e.g. with a yubikey, and then executes malicious code from this precise machine inside the valid session.
Otherwise I guess I can write a SUID script that waits for the approval of two connected users, but I was hoping that someone did it in the past…
At a fundamental level, there is no way for your laptop user to be able to create a new NixOS config for the server to use without your user having the power to take over that server. Even if you did a pull-based system, your compromised laptop can just upload a new nixos config that includes a systemd service that deletes all data, and the server will just pull it and deploy it.
You’re essentially asking the server not to trust the laptop. And in that case, it really just can’t deploy a NixOS config written by it. Or, perhaps more to the point, you’ve decided you don’t trust your laptop. And in that scenario, you simply can’t be managing your server using it. It’s not an option. If you want to be able to do that, you have to put a certain level of fundamental trust in it.
Once you’ve allowed that level of trust, something like a yubikey goes a long way to preventing the most realistic types of attacks that can be carried out against the laptop.
I disagree that we can’t build a secure server if the laptop is corrupted, my trust assumption is that at least one device (laptop or phone) is safe, and my claim is that this is enough to prevent attacks from the other corrupted party. The server/suid script I propose prevents your attack, since the phone would see the diff, detect that a malicious systemd service was added, and would block the deployment.
Thanks for the cool link, but it seems like I can’t really sign a single commit from multiple computers. Or I would maybe need to sign an empty commit above the existing one to specify that this device accepts this change?
You can also sign your commits with a yubikey. If you check the commit signatures on the server end that then means MITM attacks between your git forge and laptop cannot trick your server into installing commits that your yubikey did not sign, so this whole laptop+phone thing becomes unnecessary.
If you’re concerned about your laptop showing you commit contents that are different from the ones being sent to the forge when you’re signing them, you can indeed still do code review from the phone. Signing the commit is unnecessary from the phone end, unless you don’t trust the git forge to show you the correct commits, but at that point you still need at least two exploited devices (laptop + git forge host).
Of course, if someone exploits your phone, laptop and git forge, unfortunately, you cannot ensure that you won’t install malicious commits.
Hell, if someone exploits your backup server all bets are off.
Make your CI trigger a webhook that triggers the service.