I can’t seem to get Wireguard traffic through NixOS default firewall rules without doing full networking.firewall.enable = false;.
Currently I’ve tested the same Wireguard configuration with:
fresh NixOS install, firewall enabled: doesn’t work
fresh NixOS isntall, firewall disabled: works
fresh Ubuntu install: works
OSX: works
I do have the wg ListenPort 51820 in my allowedUDPPorts but nothing seems to be hitting the rule:
0 0 nixos-fw-accept udp -- any any anywhere anywhere udp dpt:51820
also when Wireguard interface is up almost no traffic is hitting the nixos-fw table.
FWIW I’m doing my testing using manual wg-quick up in order to have identical setup on every test machine, but didn’t have luck earlier with networking.wireguard.interfaces either.
So my question is: starting from out-of-the-box NixOS (20.03) installation, what’s the minimal setup needed to route all traffic through a Wireguard interface without disabling the firewall?
After debugging this a bit more I’ve found the culcript but I’m still unsure what would be the idiomatic way to fix this.
I noticed that the packets are actually being dropped in the raw table and the rule that’s dropping them is -A nixos-fw-rpfilter -j DROP. Once that’s clear it’s easy enough to create a rule that fixes the problem:
And indeed it works. Traffic is now allowed to go through and Wireguard works as intended. I’m quite sure I can just automate this rather easily by adding pre/post hooks to my Wireguard configuration, but that doesn’t seem intended. Looking at it more closely it seems that before enabling Wireguard traffic is going through this rule:
-A nixos-fw-rpfilter -m rpfilter --validmark -j RETURN
But Wireguard traffic seems to bypass that rule? Looking at firewall.nix it seems that there’s this: ${optionalString (cfg.checkReversePath == "loose") "--loose"}.
Turns out enabling that actually does the trick. I wouldn’t mind an explanation why, but for anyone looking for a solution to the same issue, here it is. (I’ll probably try to add it to the wiki as well)
Still not completely there though. I’d like to get this working for interfaces created with networking.wireguard.interfaces.
Those seem to create identical interface to what wg-quick created, but generate different routes. wg-quick does the fwmark trick from Routing & Network Namespaces - WireGuard but interfaces created with networking.wireguard.interfaces just seem to add new default route as well as route for the assigned ip:
1d0
< default dev wg0 scope link
3d1
< 10.0.0.0/24 dev wg0 proto kernel scope link src 10.0.0.11
While this is active I’m unable to get any packets through. By removing the default route that was added traffic bypasses Wireguard and everything works normally. If I add a rule ip route add 1.1.1.1 dev wg0 traffic does indeed successfully go through the tunnel when connecting to that ip.
Adding proper ip route add <endpoint ip> via <gateway> dev <network interface> for the endpoint allows the default route to work as intended. Now both the networking.wireguard.interfaces and manual wg-quick work as intended.
Hmm potentially seem to be running into the same issue.
What does endpoint ip and gateway get set to? I’ve tried the public ip of the wireguard server, and the ‘usual’ gateway ip with seemingly no change in behavior.
Strangely if I set allowed_ips to a set of ips that is not 0.0.0.0, then it works correctly.
