I have setup traefik on a nixos host by enabling and configuring the traefik service module (i.e. its not in any kind of container itself). I have it 1) working with other service modules on the same host, running on the host’s network by pointing traefik towards localhost and the service’s port and also 2) working with oci-containers (docker) running on their own defined docker bridge network on the host by connecting to the docker container via the unix docker socket “provider”.
I’d REALLY like to get traefik working with native nixos systemd-nspawn containers as well but have hit a seriously steep learning curve on this specific issue and have expended close to 15 hours troubleshooting and trying dozens of different configurations without success. According to the wiki entry on nixos-containers AND a blog post the wiki links to AND the nixos manual entry on declarative containers, all that should theoretically be needed is to create a veth pair with the following snippet for the container:
containers.foo = {
privateNetwork = true;
hostAddress = "172.18.1.2"; # this is the address of the veth pair on the host
localAddress = "172.18.1.3"; # this is the address of the veth pair inside the container
config = {config, pkgs, lib, ...}: {
services = ...
and then pointing traefik towards the container’s localAddress, but what is happening in my case is that I can ping the container from the host, but I’m getting a “bad gateway” error in the traefik logs.
I can’t do macvlan container networking for this, and don’t want to open any ports on the host for the container. Here is ip a command on the host:
ve-uptime-kuma@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether a6:60:30:2e:39:da brd ff:ff:ff:ff:ff:ff link-netnsid 3
inet 169.254.140.143/16 metric 2048 brd 169.254.255.255 scope link ve-uptime-kuma
valid_lft forever preferred_lft forever
inet 192.168.42.241/28 brd 192.168.42.255 scope global ve-uptime-kuma
valid_lft forever preferred_lft forever
inet 172.18.1.2/32 scope global ve-uptime-kuma
valid_lft forever preferred_lft forever
inet6 fe80::a460:30ff:fe2e:39da/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
Here is the result of curl -v http://172.18.1.3:3001 from the host:
* Trying 172.18.1.3:3001...
* connect to 172.18.1.3 port 3001 from 169.254.140.143 port 46644 failed: Connection refused
* Failed to connect to 172.18.1.3 port 3001 after 0 ms: Couldn't connect to server
* Closing connection
curl: (7) Failed to connect to 172.18.1.3 port 3001 after 0 ms: Couldn't connect to server
Can anyone help me understand what I’m missing to get this working? Here are the relevant parts of my config right now:
{
config,
configVars,
pkgs,
lib,
...
}:
let
app = "uptime-kuma";
in
{
containers.${app} = {
autoStart = true;
privateNetwork = true;
hostAddress = "172.18.1.2";
localAddress = "172.18.1.3";
config = {config, pkgs, lib, ...}: {
services = {
${app}.enable = true;
resolved = {
enable = true; # use systemd-resolved for DNS functionality inside container
llmnr = "false"; # disable link-local multicast name resolution inside container
};
};
networking = {
firewall = {
enable = true;
allowedTCPPorts = [3001];
};
useHostResolvConf = lib.mkForce false; # use systemd-resolved inside the container
};
system.stateVersion = "23.11";
};
};
services.traefik.dynamicConfigOptions.http = {
routers.${app} = {
entrypoints = ["websecure"];
rule = "Host(`${app}.${configVars.domain3}`)";
service = "${app}";
middlewares = [
"secure-headers"
];
tls = {
certResolver = "cloudflareDns";
options = "tls-13@file";
};
};
services.${app} = {
loadBalancer = {
passHostHeader = true;
servers = [
{
url = "http://172.21.1.2:3001"; # 502 bad gateway error in the traefik access log
url = "http://172.21.1.3:3001"; # also tried this, 502 bad gateway error in the traefik access log
}
];
};
};
};
}