Newbie openssh server questions

I’m trying to set up a server running NixOS. I’ve never set up a server of any kind before, although I’ve used plenty of clients.

My client (“hp17”, a Hewlett Packard laptop from 2017) doesn’t even see my server, let alone connect.

# bash from client

[jeff@jbb-hp17:~/code/phan]$ nmap 181.56.238.79
Starting Nmap 7.92 ( https://nmap.org ) at 2022-05-09 19:15 -05
Note: Host seems down. If it is really up, but blocking our ping probe
s, try -Pn
Nmap done: 1 IP address (0 hosts up) scanned in 3.08 seconds

[jeff@jbb-hp17:~/code/phan]$ ssh jeff@181.56.238.79
<Hangs until I terminate it, with no output.>

Here are all the relevant passages from the server’s config, plus maybe a few that aren’t:

# configuration.nix on server

  # Enable the OpenSSH daemon.
  services.openssh = {
    enable = true;
    passwordAuthentication = true;
	  # I'll disable this once I can connect.
  };

  # Open ports in the firewall.
  networking.firewall.allowedTCPPorts = [ 22 ];
  # networking.firewall.allowedUDPPorts = [ ... ];
  # Or disable the firewall altogether.
  # networking.firewall.enable = false;

  networking.hostName = "server-2022";
  networking.networkmanager.enable = true;
  networking.useDHCP = false;
  networking.interfaces.enp1s0.useDHCP = true;
  networking.interfaces.wlp2s0.useDHCP = true;

  users.users.jeff = {
    uid = 1000;        # for compatibility with Ubuntu
    isNormalUser = true;
    extraGroups = [
      "wheel"
      "networkmanager"
      "dialout"
      ];
  };

  # Enable the X11 windowing system.
  services.xserver.enable = true;
  services.xserver.layout = "us";
  services.xserver.xkbOptions = "eurosign:e";

  # Enable the KDE Desktop Environment.
  services.xserver.displayManager.sddm.enable = true;
  services.xserver.desktopManager.plasma5.enable = true;

I’ve already logged in on that server as jeff (the same user that I want to ssh in as from another computer), connected to the wifi network (on which other computers are sometimes hosted too), determined its IP address, and perhaps unnecessarily started sshd.service by hand:

# bash on server

jeff@server-2022:~$ curl ifconfig.me
181.56.238.79
jeff@server-2022:~$ sudo systemctl start sshd.service
[sudo] password for jeff:
jeff@server-2022:~$

I’ve tried with my client laptop on the same network or on another one. (I can connect to AWS using that client from either network, so I know that’s not the problem.) I’ve tried connecting both before running sudo systemctl start sshd.service and after, thinking maybe that procedural action interferes with NixOS’s declarative idiom.

Is systemctl start unnecessary? That is, will the SSH daemon start automatically when I log in? Can I make the daemon start before I log in, so that I can keep it offsite and if someone reboots it I can still log in from afar?

sshd should start on its own. Note that network manager:

networking.networkmanager.enable = true;

And your other network configuration:

  networking.useDHCP = false;
  networking.interfaces.enp1s0.useDHCP = true;
  networking.interfaces.wlp2s0.useDHCP = true;

are two different network management systems and generally should not be used together.

2 Likes

Adding to @emmanuelrosa 's comment, I’d suggest using dhcp, and using it on enp1s0 only for a server that’s presumably connected via ethernet. Connecting via wifi will take more configuration.

networkmanager is nice if you need flexibility in connecting to new networks all the time (e.g. for internet cafes and whatnot), which a server probably doesn’t do, so it’s probably best to not enable it in this case. It’s better for laptops, or perhaps if you have a very complex wifi network to connect to and can’t figure out wpa_supplicant.

Most NixOS services are configured to auto-start when the system boots by default if you enable them, openssh is one of them. If you want to know if a service is running, you can use, for example systemctl status sshd to check - it will give you the last couple of log lines of the service, as well as its “status” (which is most likely active for sshd in this case).

In other words, the ssh daemon has already started before you even log in. You can make your first login purely via ssh, and never need to log into the server physically. This is how servers out there on the internet can be maintained via AWS and whatnot.

systemd is what orchestrates all the service starting/stopping and collecting their logs. Being familiar with it will help a lot in understanding NixOS (and pretty much all other modern Linux distros).

Other NixOS services have autoStart or similar options.

Starting/stopping systemd services is well within the things you’re expected to do while using NixOS :slight_smile: Yes, in this case it’s unnecessary to start openssh, but don’t be afraid of the systemctl commands, just don’t use systemctl edit - that will change configuration files imperatively, and will therefore make your configuration and actual running system mismatch.

You may also want to know about journalctl --catalog --pager-end --unit=sshd (or short journalctl -xeu sshd), which will give you the complete logs of your sshd service, scrolled to the bottom, and with “catalog” information about when the service started/stopped.

Even if you do accidentally change files, NixOS functions just like any other Linux distribution. You just end up with mismatches between your declarative config and the real system, which may bite you when you update, but should have no immediate effects. Don’t be afraid to experiment to get a feel for what you can and can’t do, in the worst case reinstalling a NixOS config is much easier than that of traditional distros.

If you’re very paranoid of breakage, maybe give Erase your darlings: immutable infrastructure for mutable systems - Graham Christensen a try. That kind of setup makes it very hard to break NixOS’ model of your system.

To the actual problem - your server config looks good, except for the aforementioned double network configuration. That might be interfering with this.

If it’s not the problem, I think your problem is on the network rather than server level. Where are you connecting from? Are both your devices on the same intranet, connected to the same router? Does your router have firewall settings that block these kinds of connections? Are you connecting from outside to a residential network and need to enable port forwarding? Is the IP you’re using actually correct?

If your router is configured to allow this, you may in fact be able to use the hostname for ssh: ssh jeff@server-2022.

You’re trying to reach the server over its public IP address that may or may not be shared with your entire network via NAT.

You need to forward port 22 and an internet connection without CGNAT in order to be able to reach your server from your network’s public IP.

I would highly recommend against exposing ssh to the entire internet though and would recommend to use a solution like Tailscale or ZeroTier instead.

3 Likes

A sufficiently well-protected home NAS with an open SSH port is probably not the end of the world, but outsourcing the security aspect of your connection is probably not a bad idea either if you’re not too sure what you’re doing.

1 Like

Wow, this is more complex than I expected.

Disabled networkmanager, failed at wpa_supplicant

I changed the networking options in my config to now read:

  networking.hostName = "server-2022";
  networking.networkmanager.enable = false;
  networking.useDHCP = false;
  networking.interfaces.enp1s0.useDHCP = true;
  networking.interfaces.wlp2s0.useDHCP = true;

I also installed wpa_supplicant and wpa_supplicant_gui. Once running the new configuration, I manually (is that bad?) created this file:

# /etc/wpa_supplicant/wpa_supplicant.conf
ctrl_interface=DIR=/var/run/wpa_supplicant
update_config=1

then started wpa_supplicant as sudo:

[jeff@server-2022:~]$ sudo wpa_supplicant -c /etc/wpa_supplicant/wpa_supplicant.conf -i wlp2s0
[sudo] contraseña para jeff:
Successfully initialized wpa_supplicant

and finally started wpa_gui as sudo:

[jeff@server-2022:~]$ sudo wpa_gui
[sudo] contraseña para jeff:
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'

All that imperative stuff is based on this Ask Ubuntu answer.

The result: Scanning for networks in wpa_gui finds nothing.

My connection is probably CGNAT

The server is on a home wifi network. When I check its public IP address (by running curl ifconfig.me), and then check the public IP address of another computer using the same wifi router, I get the same value. The term CGNAT is new to me but a little reading suggests that it’s generally the default. I therefore suspect I’m on a CGNAT connection.

I’m in Bogotá, Colombia. In my neighborhood neither power nor internet are completely stable. My plan was to keep the server at a friend’s place, because he lives in a fancier neighborhood with better services.

If I were going to test the connection out here before moving it there, do I understand correctly that I’d have to talk my ISP into giving me my own public IPv4 address, and do the same to his ISP (maybe they’re the same, I don’t know)? That in itself could be hard.

I’d like to use wifi

instead of plugging a cable into the computer. My router is hard to set up a computer next to, and my friend’s place is similar.

No, that’s just regular NAT so far. Every residential connection has NAT’d IPv4.

It’s likely that you are behind CGNAT too but you’d have to check. Some places aren’t.

You’d have to set your router to forward a certain port to your server. If you can’t do that, that’s end of the line.
You should then also check your router’s IP of the interface facing towards the internet. If it’s also 181.56.238.79, you’ve got your own IPv4 and are not behind CGNAT. If it’s a private IP address, you’re behind CGNAT.

Anyhow, Tailscale might be a good idea anyways since you’re unsure about your network condition at the place you’ll keep the machine.

instead of plugging a cable into the computer. My router is hard to set up a computer next to, and my friend’s place is similar.

Frankly, that’s a terrible idea. Just plug in a cable.

Better in every aspect.

Not per-se, as @Atemu says it might not be a CGNAT. Even if it is, you can test your server at home with your intranet address without really losing anything, since the network configuration will change anyway (ip addr show will tell you that for each network interface). You might need to talk to the ISP on your friend’s end though.

While I agree that air is a poor conductor, and copper is a much better idea, pragmatism does sometimes override that. NixOS has pretty good static configuration for that as well, see networking.wireless.enable and networking.wireless.networks.

Give that a stab and show us the config you end up with, I’m happy to help figure out the issues resulting from there. If you installed the wpa_supplicant utilities with nix-env, uninstall them again. Using nix-env on nixos will bring you pain.

That said, using ethernet will be faster, more secure, and easier. A really long cable, as long as it can be secured reasonably well (maybe across a ceiling!), isn’t expensive and easier to route than you’d think.

1 Like