Using full-disk encryption on a dedicated server

I run NixOS on a Hetzner dedicated server which was installed from the Rescue system using this script, which sets up the disks using RAID1 and LVM.

I’d like add full-disk encryption to this install script. The NixOS manual recommends LUKS … but this will have the user manually enter the encryption passphrase during boot time in front of the actual console (neither physically nor virtually accessible to me; at least with Hetzner) - something that’s not directly possible when renting a dedicated server from another country.

In several places it is being recommended to use a SSH service during initrd boot (see the corresponding Nix config) - but I do not know what the full setup would be? Apparently, you would ssh to the still-being-booted machine to manually enter the passphrase, somehow triggering the boot process to complete.

Question: how do you do this in Nix? More specifically, how would you patch the aforementioned install script to achieve this automatically? More generally, it would be nice if all this is automated for reuse from nixpkgs …

1 Like

I believe you would only need to flip that setting to on where the script defines the host’s configuration.nix and that would basically do the trick. Assuming you install your ssh certificates this will give you any access you need in that state, though you might need to experiment a bit with the user configuration to get something actually safe.

You would want to edit the script to create a luks partition somewhere around here too, of course, probably looking something like what I do on my local machine configurations.

Once you’d written that you could commit it to that script with an option flag and push up a PR. Is this what you are missing for the full picture?

All of that said, I do wonder what your goal is here; full disk encryption on a hetzner server isn’t particularly useful. It will presumably spend most of its life turned on with its disks decrypted, where it is much more vulnerable to having data exfiltrated (be it through malicious firmware, injections into the actual running memory, network monitoring, what have you). Hell, it’s trivial to install a malicious kernel into the non-encrypted boot partition that sends your encryption key to a location of Hetzner’s choice if it is turned off, unless you have secure boot, which you don’t and NixOS doesn’t currently support.

My goal is to prevent Hetzner personnel (or anybody in the vicinity of the server capable of physically taking the SSD out) from being able to peek into my actual files, at least those in /home (if not full disk).

much more vulnerable to having data exfiltrated (be it through malicious firmware, injections into the actual running memory, network monitoring, what have you).

You are saying that even full disk encryption wouldn’t help with the above goal? Yes, looking at actual running memory is one vector - but is that something I should be worried about in practice here?

I’m trying to say that that vector is much easier to abuse for the hardware owner, so be aware of your mitigations for it too if you want to go down this route. There are some things you can do to mitigate that, although I think if you’re using a general purpose Linux distro like NixOS you’re probably not mitigating it.

That said, even if you only booted your server a single time on Hetzner hardware, and then turned it off forever, disk encryption is easily broken by someone with full control over your hardware.

The problem is that your host cannot magically start an encrypted binary. Something must decrypt your drives first.

The way this is done in the Linux world is that you have a stage 1 with an “initial ram disk”, which is just a snapshot of a running kernel with an init script that knows how to decrypt your partitions. This can provide what you’re looking for if you can be sure that this initial ram disk is never tampered with - if you cannot, any initial ram disk can be inserted, and once you type in your decryption password this initial ram disk can do whatever it wants with your files and encryption key. On every boot, Hetzner could collect your password and store it.

There is a mitigation for this, albeit one not supported on NixOS (yet). With UEFI Microsoft introduced secure boot, which uses the motherboard firmware to check if the booted binary is signed by a key that you can install. This works somewhat well in consumer hardware, where you can kind of trust the motherboard manufacturer to not be malicious, and their firmware to not be blatantly buggy or backdoored (well, maybe). However, for a server provider like Hetzner this is almost useless since they control all the hardware including the motherboard fully.

I suppose this might protect you from a rogue employee who isn’t performing a premeditated attack and doesn’t have access to monitoring equipment (since it involves capturing your password while you reboot the system, which may be rare), but ultimately if you do not trust Hetzner with this data you probably shouldn’t be hosting there in the first place. It’s too easy to bypass basic disk encryption for them; whether by inspecting the running system, adding malicious devices or tampering with something to exfiltrate your password on boot.

This may match your threat model, I’m just pointing out that this may only give you a false sense of security. Still going down this path because it’s kind of cool to play with and might give you some protection is not an issue, of course, but there is a big caveat that you should not at all trust it :slight_smile:

3 Likes

I made such a config for ssh with Hetzner cloud for the play.

@TLATER is right, after reading many post on stackoverflow, security is a mirage when it concerns machine you cannot control physically (A RPI 4 for example). I migrate my data i want to protect on an encrypted SSD with RPI4 nixos connected to my VPS by tunnelling (tailscale or equivalent).

Grub manage LUKS decryption but you need an access to console because that happen before ssh, see Full encrypted nixos system on legacy boot with secrets and remote unlock, for unstable 20.03

You can still use LUKS, but not encrypt the full disk: the machine must be able to at least load a Linux kernel and set up the network autonomously. So, you can encrypt everything except the bootloader/ESP and enable the copyKernels option, so you don’t need a mounted /nix/store to boot.

The setup is described in the NixOS wiki here.