How to use two WAN connections in NixOS

Help
1

I’ve got two internet connections, both going into one NixOS box. In the olden days (before two PFSense boxes died in one day), I might have used PFSense, and gotten simple failover Wan handling. No extra speed, but of one of them failed, the system would switch over to the other one, and back when the first recovered. Now, I’m crazy enough to try to do it in NixOS, without knowing exactly what I’m doing. I’m happy to learn, and I got excited when I saw the mptcp-flake github repo, but I realized I haven’t the first idea how to put that into practice.

So what I’m looking for is one of:

  1. Two wan connections, either one capable of serving whatever traffic it gets. No worries about failover in this model, and any outgoing will just use whichever wan is declared the default route. The only issue here is having the reply go out on the same WAN it came in on.

  2. All traffic comes in and out on one WAN, but if it fails, the system (within a minute or two) switches to the other WAN. Eventually, when the first WAN recovers plus a minute or two, it switches back.

  3. Any other cool way to use two WAN connections without using a single proxy in the cloud to route all traffic.

    Any way to help a network Noob?

3 Likes
2

haha interesting, that’s one usecase for MPTCP. The issue is that the server needs to understand MPTCP. One way to work around that is to have a proxy in the cloud to terminate your MPTCP connection as is shown in GitHub - Ysurac/openmptcprouter: OpenMPTCProuter is an open source solution to aggregate multiple internet connections using Multipath TCP (MPTCP) on OpenWrt

I created this repo to test some of my code. The module should work but I am ready to help you troubleshoot issues (just open issues directly in the repo).

2 Likes
3

I may have to try it, for sheer curiosity, but I didn’t know I’d have to send everything through a proxy to make that work. (Editing the question to emphasize the wan failover, or even just having each WAN serve whichever requests it receives). Thanks!!!

4

You may also try out Shorewall package and follow their extensive docs

5

I looked at their extensive docs and read left with two questions. Since the release notes for 22.05 strongly hint that any firewall not built on nftables might break, does shotwall work well enough? Also, is there a repository of examples for it?

Alternatively, has anyone done it in nixos, by any means, and have a config to learn from?

6

Really? Where? I found that iptables uses iptables-nft under the hood but nothing else…

1 Like
7

I suppose you can do all with ip Routing for multiple uplinks/providers

8

Actually I got the release wrong: it was 21.11. Teach me to scroll without looking.

B.2.3. Backward Incompatibilities

… other stuff removed …

  • The security.wrappers option now requires to always specify an owner, group and whether the setuid/setgid bit should be set. This is motivated by the fact that before NixOS 21.11, specifying either setuid or setgid but not owner/group resulted in wrappers owned by nobody/nogroup, which is unsafe.
  • Since iptables now uses nf_tables backend and ipset doesn’t support it, some applications (ferm, shorewall, firehol) may have limited functionality.
9

ipset is seldomly used by shorewall especially to do (auto) blacklisting of IPs, so if you don’t use such features probably it will work but if anything will go wrong during the startup it will tell you that :wink:

10

Awesome!!! I’ll try it next week!

11

I am curious if using the flake worked for you since I am also trying to do a multi-ISP setup, but I am having issues figuring out how to get it to switch between the connections

12

Nothing I tried worked, so I got rid of the second WAN connection.

13

what did you try just out of curiosity?

14

I’d like to try using bonds with systemd-networkd personally, though I’ve not yet got around to it. They seem like they’d work for this.