The relevant part of my configuration.nix is below:
services.openssh = {
enable = true;
settings.PasswordAuthentication = true;
};
networking.firewall.enable = false;
The relevant part of my configuration.nix is below:
services.openssh = {
enable = true;
settings.PasswordAuthentication = true;
};
networking.firewall.enable = false;
If you are connecting your laptop to your network using wifi, then it’s likely a setting on your wireless router enforcing wifi “client isolation” that is the cause of this. You can test this by trying to connect to your NixOS machine from a computer connected to your network using ethernet or connect your latop to ethernet if able.
If you are already using an ethernet connection for networking, we will need more information to be able to help. Exactly what error message do you receive on the laptop when you try to connect to the NixOS machine. Is the ssh server running properly on the NixOS machine, what is the output of systemctl status sshd.service
. Can you at least ssh from the NixOS machine into itself (ssh 127.0.0.1
).
When I try to ssh it just times out, it’s as if the NixOS machine or it’s IP don’t exist. The ssh service is running on the NixOS machine as well. Client isolation is also off on my router:
Well, an easy test to determine if there is indeed a running service listening and responding is by using nmap with the -sT and -sV flags, or -sS -sV.
I dont expect you to read the entire nmap source manual if you arent already familiar, so heres a cut paste command you can use to verify a running SSH server is listening on the other end.
Grab a nix shell with nmap
nix-shell -p nmap
then (as sudo or root, port 22 with the -sS flags require raw socket permissions)
sudo nmap --privileged -sS -sV -p 22 -Pn -vv --reason 127.0.0.1
Or if this is a production/managed env where sudo is not coo for whatever reason, you can use this which doesnt require sudo or raw socket access, just know its usually a little less accurate under normal conditions, but i expect you probably wont have issues.
nmap -sT -sV -p 22 -Pn -vv --reason 127.0.0.1
Should return something like this:
[jason@Beast:~]$ nmap -sT -sV -p 22 -Pn -vv --reason 127.0.0.1
Starting Nmap 7.95 ( https://nmap.org ) at 2025-01-13 20:59 CST
NSE: Loaded 47 scripts for scanning.
Initiating Connect Scan at 20:59
Scanning localhost (127.0.0.1) [1 port]
Discovered open port 22/tcp on 127.0.0.1
Completed Connect Scan at 20:59, 0.00s elapsed (1 total ports)
Initiating Service scan at 20:59
Scanning 1 service on localhost (127.0.0.1)
Completed Service scan at 20:59, 0.01s elapsed (1 service on 1 host)
NSE: Script scanning 127.0.0.1.
NSE: Starting runlevel 1 (of 2) scan.
Initiating NSE at 20:59
Completed NSE at 20:59, 0.00s elapsed
NSE: Starting runlevel 2 (of 2) scan.
Initiating NSE at 20:59
Completed NSE at 20:59, 0.00s elapsed
Nmap scan report for localhost (127.0.0.1)
Host is up, received user-set (0.000066s latency).
Scanned at 2025-01-13 20:59:07 CST for 0sPORT STATE SERVICE REASON VERSION
22/tcp open ssh syn-ack OpenSSH 9.9 (protocol 2.0)Read data files from: /nix/store/ny1b8vadzvpr44d8s9z7v3qiyda18r8f-nmap-7.95/bin/…/share/nmap
Service detection performed. Please report any incorrect results at Nmap OS/Service Fingerprint and Correction Submission Page .
Nmap done: 1 IP address (1 host up) scanned in 0.17 seconds
you can also get the banner of a service thats running with out probes using the scrit engines prebuilt banner module which will help narrow down versioning info if need be.
nmap -sT -sV -p 22 -Pn -vv --reason 127.0.0.1 --script=banner
Run this for both machines, the client towards the server and the server towards itself. This will tell you where the problem is occurring, which will definitively rule out firewall or black holing issues.
If the server responds to itself but not the client, that means its likely a firewall problem on the router.
If the server doesnt respond to itself or the client, it might be a firewall issue on the SSH server box, dump your FW rule table or use wireshark to see what the system is doing with the packets, or simply turn it off and test again.
It returned the following:
...
Host is up, received user-set (0.000036s latency).
...
PORT STATE SERVICE REASON VERSION
22/tcp open ssh syn-ack ttl 64 OpenSSH 9.9 (protocol 2.0)
Did you run this from the client or the server?
From the nixos server
Run nmap from the client to the server and paste the results, run the command with extra debug flags like so:
nmap -sT -sV -p 22 -Pn -vv --reason 127.0.0.1 -dd --script=banner
Copy paste the whole output if you can
Completed Connect Scan at 22:12, 2.01s elapsed (1 total ports)
Overall sending rates: 1.00 packets / s.
Fetchfile found /usr/local/bin/../share/nmap/nmap-service-probes
Initiating Service scan at 22:12
NSE: Script scanning 192.168.200.64.
NSE: Starting runlevel 1 (of 2) scan.
Initiating NSE at 22:12
Completed NSE at 22:12, 0.00s elapsed
NSE: Starting runlevel 2 (of 2) scan.
Initiating NSE at 22:12
Completed NSE at 22:12, 0.00s elapsed
Nmap scan report for 192.168.200.64
Host is up, received user-set.
Scanned at 2025-01-13 22:12:44 EST for 2s
PORT STATE SERVICE REASON VERSION
22/tcp filtered ssh no-response
NSE: Script Post-scanning.
NSE: Starting runlevel 1 (of 2) scan.
Initiating NSE at 22:12
Completed NSE at 22:12, 0.00s elapsed
NSE: Starting runlevel 2 (of 2) scan.
Initiating NSE at 22:12
Completed NSE at 22:12, 0.00s elapsed
Read from /usr/local/bin/../share/nmap: nmap-protocols nmap-service-probes nmap-services.
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 2.31 seconds
Not a timeout, the packets are getting sink-holed in transit.
What type and model of router do you have?
The router is a ZyXEL VMG4381-B10A
From what i can tell, according the manual and docs for your router, the default config should allow for SSH port 22 on LAN without restriction so i suspect the issue lies within the prebuilt nixOS firewall which is enabled by default and fairly restrictive. Doesnt use PF like older unix machines did, uses iptables and im not totally familiar with the syntax on writing specific rules but we can see if the FW really is the problem by disabling it for a minute on the server machine, running a rebuild and trying nmap/ssh again from the client machine.
Set this in your ‘/etc/nixos/configuration.nix’ (on the server)
networking.firewall.enable = false;
and try again
The firewall has been disabled this whole time. I agree, the router shouldn’t be the issue, I’ve been able to ssh onto other machines over the local network in the past, but this NixOS install is the only one that’s giving me issues, not sure why, it seems rather straightforward (basically doing what I would do with commands in a config file)
Hmm. That is strange. Alright lets dig a little deeper. Use the same nmap command i gave you earlier from server to itself but instead of targeting 127.0.0.1, target the LAN ip address its assigned from the routers DHCP server and see if you get the same no-response reply from port 22.
Running sudo nmap --privileged -sS -sV -p 22 -Pn -vv --reason 192.168.200.64
on the NixOS machine with the local IP 192.168.200.64, I get the same thing as before:
...
Host is up, received user-set (0.000036s latency).
...
PORT STATE SERVICE REASON VERSION
22/tcp open ssh syn-ack ttl 64 OpenSSH 9.9 (protocol 2.0)
Alright so they are def getting dropped in transit by the router then, assuming you dont have any other network equipment plugged in between the 2.
When you run SSH client, can you try again with debug flags enabled? Does it give you any useful data at all?
A no-response [filtered] status is a generic translation of “probably being dropped or rate limited by FW”
Open is open of course, no problem all green
And closed-rst would indicate the system received the message, and decided to refuse the connection all together and send a reset packet which isnt whats happening here. The server sees itself, but the client doesnt.
So lets see if the client can see the machine at all on the network.
Run:
sudo nmap --privileged -n -vvv --reason --top-ports 500 -sSV --script="default and discovery" -T5 192.168.200.64
ssh -vvv <user>@192.168.200.64
output on client:
OpenSSH_9.7p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/sam/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug2: resolve_canonicalize: hostname 192.168.200.64 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/sam/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/sam/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: channel_clear_timeouts: clearing
debug3: ssh_connect_direct: entering
debug1: Connecting to 192.168.200.64 [192.168.200.64] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: connect to address 192.168.200.64 port 22: Operation timed out
ssh: connect to host 192.168.200.64 port 22: Operation timed out
nmap output on client:
Starting Nmap 7.95 ( https://nmap.org ) at 2025-01-13 22:42 EST
NSE: Loaded 135 scripts for scanning.
NSE: Script Pre-scanning.
NSE: Starting runlevel 1 (of 2) scan.
Initiating NSE at 22:42
Completed NSE at 22:42, 0.00s elapsed
NSE: Starting runlevel 2 (of 2) scan.
Initiating NSE at 22:42
Completed NSE at 22:42, 0.00s elapsed
Initiating ARP Ping Scan at 22:42
Scanning 192.168.200.64 [1 port]
Completed ARP Ping Scan at 22:42, 1.43s elapsed (1 total hosts)
Nmap scan report for 192.168.200.64 [host down, received no-response]
NSE: Script Post-scanning.
NSE: Starting runlevel 1 (of 2) scan.
Initiating NSE at 22:42
Completed NSE at 22:42, 0.00s elapsed
NSE: Starting runlevel 2 (of 2) scan.
Initiating NSE at 22:42
Completed NSE at 22:42, 0.00s elapsed
Read data files from: /usr/local/bin/../share/nmap
Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn
Nmap done: 1 IP address (0 hosts up) scanned in 1.89 seconds
Raw packets sent: 2 (56B) | Rcvd: 0 (0B)
Ok I think I may have kinda sorta figured it out. I connect the NixOS server to ethernet and was able to ssh into from the client, so it must be some issue with it being connected over wifi. Unfortunately I need to use that ethernet drop for something else in the house, so ideally I’d like to get ssh to work over wifi.
OH. It all makes sense now. Your router WiFi function by default likely puts wireless clients on a subnet thats non routable or sink holed. Some do this for a reason i’ve never fully understood.
Try setting your wifi access point setting from infrastructure mode or adhoc, whichever it is to just access point, and if you know how, stick all your DHCP client table leases to the same subnet to ensure they are routable and reboot your router.
try again