It is times like these when I wish I was more confident in my ability to do network things. Let me lay out the story so far.
I’m using NixOS as a server. That server runs containers.
virtualisation.docker.enable = true;
I have setup macvlan networking as an networking option for my docker containers. This allows me to segment part of my IP address range to be effectively ‘assigned’ to given containers. My blog post on docker + macvlan
Part of my docker network setup looks like this
networking.macvlans."macvlan-shim" = {
mode = "bridge";
interface = "enp4s0";
};
# Add routes for the macvlan containers.
# This includes magic to route to the wireguard container running on 192.168.1.48
networking.interfaces."macvlan-shim".ipv4 = {
addresses = [{ address = "192.168.1.55"; prefixLength = 32; }];
routes = [{ address = "192.168.1.48"; prefixLength = 29; } { address = "10.13.13.0"; prefixLength = 24; via = "192.168.1.48"; }];
};
I have ‘reserved’ the 192.168.1.55 IP as my pathway from the host - to the macvlan based docker containers. The first route { address = "192.168.1.48"; prefixLength = 29; } allows that any traffic going to those containers to flow from the host, to the container on that ip address.
Thus - traffic from my host - 192.168.1.88 can reach 192.168.1.48 which is a container running on that host. That container is running wireguard.
The second ‘route’ { address = "10.13.13.0"; prefixLength = 24; via = "192.168.1.48"; } is there to allow me to reach from the host (192.168.1.88) – clients of the wireguard container that is running at 192.168.1.48.
It might be easier to explain by showing the output of ip route
192.168.1.48/29 dev macvlan-shim proto static scope link
10.13.13.0/24 via 192.168.1.48 dev macvlan-shim proto static
The above is all working - on the host machine, I can ping a client that is remote and connecting to my wireguard container with the IP of 10.13.13.2. Magic.
For my next trick, I want to connect from another container running on the host, but access a specific port on a remote client connected to my wireguard container.
What I’ve done in the past - is just to setup a ssh tunnel on the host - because as I’ve stated above, the ip route setup I have – makes this possible. I use autossh to do this
autossh -M 20000 -f -L 0.0.0.0:8100:10.13.13.2:8100 -L 0.0.0.0:8101:10.13.13.2:8101 -N -T user@localhost
Since my docker containers can see the host - they are quite happy to hit port 8100 - the ssh tunnel I’ve created causes that connection, to reflect over to the client connected to my wireguard (also running in a container) on IP 10.13.13.2 port 8100
It feels like I should be able to configure port forwarding to do this… but the details here elude me. It does seem like using ssh is a horrible hack, so I’d like to stop doing that.
Yes - while I could just run wireguard ‘natively’ on NixOS, I’m happy using the container. I really like the set of containers that come from linuxserver.io overall.