Nix homelab feedback

Hi Nix Community :slight_smile:
I recently got into Nix to manage my “homelab” on Hetzner.
All configuration is available here.

I would really appreciate any feedback on structure, best practices, security, or general Linux server recommendations. I would also like to keep the repo public for any other newcomers. Do you think this makes my server more vulnerable? The data in Paperless should remain safe.

Thank you very much in advance!

Hello
So your config seems pretty good. For security reasons I would not make the firewall config public. With that you are giving away which ports you have open. Also not make your user config (especially your ssh-key) private. Also your root password should not be displayed in public even if it is hashed. Even if it is long and safe. Also blur out everything regarding ip adresses.
Get rid of the following in your public reoo:

    firewall.enable = true;
    firewall.allowedTCPPorts = [
      80 # nginx http
      443 # nginx https
      3851 # ssh
    ];
  }; 
 users.users.www = {
    isNormalUser = true;
    home = "/var/www";
    group = "www";
    shell = pkgs.dash; # rrsync bash security issue
    openssh.authorizedKeys.keys = [
      # append only key
      ''command="${pkgs.rrsync}/bin/rrsync /var/www/homepage",restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFky8zixRqFyQVNlykhWdt4gp1XMi2DATurTgWpuayu4''
    ];
  };
  system.activationScripts = {
    scriptChrootJail = {
      deps = [ "specialfs" ];
      text = ''
        mkdir -p /var/www/
        mkdir -p /var/www/homepage
        chmod 755 /var/www/
        chmod 755 /var/www/homepage
        chown root:root /var/www/
        chown www:www /var/www/homepage
      '';
    };
  };
  users.users.behrn = {
    isNormalUser = true;
    extraGroups = [
      "networkmanager"
      "wheel"
      "keys"
    ];
    packages = with pkgs; [ ];
    openssh.authorizedKeys.keys = [
      "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIbUF1O0ajT8PFQeUyNsJJLjl5P6ByRKI+JlXR1apimR"
    ];
  };
  security.sudo.extraRules = [
    {
      users = [ "behrn" ];
      commands = [
        {
          # needed for colmena
          command = "ALL";
          options = [ "NOPASSWD" ];
        }
      ];
    }
  ];

  # ROOT
  users.users.root.hashedPassword = "$y$j9T$gwUVzCIiyNDk5Ybtjtfep.$yaqYMsIBQMMj/5AS92p3WWkpIAuEdHp6T8YEh5ORjl/";

Either you delete those ore blur them out. These can be a massive security risk especially on a server and a public repo. When you aim for a repo for new users blur them out.

Also I would recommend you to use failtoban (Seen you already did, but you can decrease after how many times an IP address gets banned to something like 2-3) and maybe use appamour.
For some more info you can look there:

It does. You are exposing quite a lot of information and potential attack vectors.

Deciding what to keep under version control - and how - depends on your threat model (Harvest now, decrypt later - Wikipedia). Personally I’d rather err on the side of caution, especially if you plan to use your config as a teaching tool.

Did you already have a look at secret management tools (agenix, sops-nix)? Even then, I’d not commit encrypted secrets to a public repo.

Also hide your ssh port.

This all seems like security theater to me. Ports are easy to scan, ssh public keys are by definition safe to publicize unless you’re worried about quantum computers, password hashes are safe to publicize as long as the salt is unique, and IP addresses are basically public info already.

Basically all of your suggestions are known as “security through obscurity”, not actually useful security advice. These seem like things that can provide a very soft, minimal security barrier. And I mean, it’s fair to say that even soft barriers are an improvement to security, but it’s not by very much.

9 Likes

I defenetly see you point. But since this is an open Repo that should help new comers to get started on, I think it is best to not show this info. Because even if it is

This info should not need to be public and it is important to show that to new users.

Since you obviously have some background in cyber security, I think it would be more helpful for @NicholasBehr if you provide some feedback for him.

The point is basically that your suggestions boil down to maybe 5 minutes of extra work for a dedicated attacker, and zero for the usual broad scanning attack that is most likely to be an issue for a random homelab as it would not bother trying to match up random NixOS configs to your server, and just do a much cheaper scan on public data instead. Without a good cloudflare/fail2ban/crowdsec config it probably doesn’t even buy you 5 minutes.

With the exception of the hashed password all of that data is public by design. I think not to include the hashed password is fair, since that at least is something a dedicated attacker would not be able to get as easily elsewhere. But it indeed also is fairly safe, since you’d need some serious work to crack that anyway, particularly if you rotate it regularly and don’t permit password login on ssh. Still, my preference is to treat secrets as data, since rotating a secret should not require config changes.

If you prioritize the sharing aspect, keeping a config like this is totally fair. Even if not, this doesn’t pose a significant risk by itself. Just don’t go sharing your secrets in plaintext.

2 Likes

Wow, thank you all for the input. I really appreciate every one of you!

// Dump of my thoughts
I was also under the impression that exposing public keys should be fine as long as there is no misconfiguration. The ideal would be to have a config that makes the server safe, even if this config is exposed (which should have been my question). What would that take here e.g. @ElvishJerricco @TLATER? The changed SSH port is only to ward off basic bots and is no real security measure in my books. The (hashed) root password shouldn’t ever be a problem since both the password ssh and root ssh are blocked, right? (su root might be a challenge down the line)? I only plan it to be used in the “local” hetzner terminal if I screw up ssh.

Basically, it seems to boil down to the SSH keys. I see how exposing them makes it possible to attack them offline, which happened to past (now) weak ssh ssh ciphers.
And I am conveniently also disclosing the login user name …
If I am at it, I would also include the hashed password.

// Question
How could I put these in a variable and assign them in a separate file that I won’t check into version control? My Nix skills aren’t there yet.
Or what would be the best practice to get them out of the version control?

So, I’m opinionated about these things because anyone who spends time administering servers will be, but my 2c:

… unless an attacker exploits a vulnerability in an exposed service that is otherwise unprivileged. This is always the main risk outside of leaking secrets or otherwise accidentally allowing unauthorized access. Even then, many NixOS services are “hardened” and won’t permit unprivileged services elevating their privileges, though this isn’t completely bulletproof.

Personally, I just don’t set root passwords (and thereby prevent root login) and use an account with wheel permissions that can only be signed into via a public key, which I then use for root auth with pam. For recovery, you can always boot a live distro and chroot into the system, so root passwords are practically useless and should therefore be disabled IMO.

In practice this probably doesn’t change much; whether sudo protects the system or if the kernel guards it is splitting hairs, if anything the ssh agent adds a potentially weak link. And again, your homelab setup is unlikely to fall to someone paying hundreds of thousands to crack a hashed password - my setup exchanges that small weakness for the chance of bugs in pam.

You can always add the key to the ~/.ssh/authorized_keys file by hand, or use something like nixos-anywhere to install it at initial deploy time, then develop some deployment strategy to update it when needed (or just add another line by hand whenever you rotate, and delete the old one after you confirm you can log in with the new one).

This is an incredibly unlikely way to gain entry into your system, though. If you’re worried about quantum computers recovering your private key, use an ECC key, and use key expiry so you rotate regularly. Guarding public keys is incredibly paranoid, they’re called public for a reason - if this breaks, you’ll have bigger problems than your homelab.

There are a few ways to achieve this, but IMO the best way is to treat your secrets as simple data and deploy them without the NixOS config. If you want to use the NixOS config, for the root password you can use hashedPasswordFile to point at a file that is somewhere on the host, and then use something like nixos-anywhere or deploy-rs to copy the file in the right place when you deploy. Then just don’t check it into version control.

You can also use mutableUsers and passwd.

For the public key you would need to write your own systemd service to put it in the right location if you wanted to achieve this. It’s considered public, so the NixOS modules have no built-in support for hiding it.

This concept can be enhanced with sops-nix or agenix, which provide a generic interface for this kind of setup. They also encrypt the secrets at rest. Note that neither requires checking the encrypted secrets into VC, you can just leave them backed up on a USB or something, and again deploy them to the host without nix when needed - data, not config!

With your simple setup, though, I don’t think you need this. The added complexity from this just adds risk - stick to passwd if you’re paranoid, disable root login altogether and use pam, or keep the hash in your config.

2 Likes