Hi,
I have a problem with DNS resolution in a multinode kubernetes kind cluster. The reason is the default NixOS firewall. If I disable the firewall, DNS works. But I do not want to disable it completely. What would be a better way to configure the firewall?
To reproduce the issue follow these steps:
Create config file:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
Create cluster
kind create cluster --config kind-cluster-config.yaml
Test DNS:
kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml
kubectl exec -it dnsutils -- dig kubernetes.default.svc.cluster.local
; <<>> DiG 9.16.27 <<>> kubernetes.default.svc.cluster.local
;; global options: +cmd
;; connection timed out; no servers could be reached
command terminated with exit code 9
When I enable rpfilter logging I see the following messages in journalctl -k:
Feb 28 11:09:12 jupiter kernel: rpfilter drop: IN=br-d36dd023e4ab OUT= PHYSIN=veth25bef20 MAC=02:42:ac:12:00:02:02:42:ac:12:00:03:08:00 SRC=10.245.1.3 DST=10.245.0.3 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=16754 DF PROTO=TCP SPT=38102 DPT=53 WINDOW=64240 RES=0x00 SYN URGP=0
The workaround:
set networking.firewall.checkReversePath = false; in configuration.nix