Routed network kvm

Help
1

Has anyone managed to get routed networking to work with KVM on NixOS?

Using TCP dump, I can see that I am getting the traffic reaching the Vm , but it never actually makes it in.

I’m essentially hoping to have a number of KVM networks in their own subnets. But being routable to my real network. thank you.

Now one thing to Notes here. I’m hoping to figure this out using network manager, and not using network manager.

The reason is, I have KVM running on both my desktop, plus a server.

2

Another attempt

{ config, pkgs, lib, inputs, ... }: {
  boot.kernel.sysctl = {
    # Enable IP forwarding
    "net.ipv4.ip_forward" = 1;
    # controls whether packets traversing a Linux bridge will be passed through iptables' FORWARD chain. When set to 1 (enabled), it allows iptables rules to affect bridged (as opposed to just routed) traffic.
    "net.bridge.bridge-nf-call-iptables" = 1;
  };

  networking = {
    nat = {
      enable = true;
      externalInterface = "br0"; # Your external interface
      # Note
      # - for every routed network created in Terrraform, you need to add a new internal interface here
      # - and a static route needs to be added to the LAN router for the new network
      internalInterfaces = [ "virbr1" "virbr2" "virbr3" "virbr4" "virbr5" "virbr6" "virbr7" ]; # Your KVM bridge interface
    };
    firewall = {
      enable = true;
      allowedTCPPorts = [ ]; # Empty since we're allowing all traffic
      allowedUDPPorts = [ ]; # Empty since we're allowing all traffic
      extraCommands = ''
        # Allow all incoming and outgoing traffic on internal LAN interface
        # iptables -A INPUT -i virbr3 -j ACCEPT
        # iptables -A OUTPUT -o virbr3 -j ACCEPT
      '';
    };
  };
}

This is just routing on my LAN, hence I am not blocking anything thus far. This is just a POC, and wanted to share.

3

spoke too soon. FOr some reason only SSH works. No other traffic.

Current config.

{ config, pkgs, lib, inputs, ... }: {
  boot.kernel.sysctl = {
    # Enable IP forwarding
    "net.ipv4.ip_forward" = 1;
    # controls whether packets traversing a Linux bridge will be passed through iptables' FORWARD chain. When set to 1 (enabled), it allows iptables rules to affect bridged (as opposed to just routed) traffic.
    "net.bridge.bridge-nf-call-iptables" = 1;
    "net.ipv4.conf.all.forwarding" = 1;
    "net.ipv6.conf.all.forwarding" = 1;
    "net.ipv4.conf.all.proxy_arp" = 1;
    "net.ipv4.conf.ens2.proxy_arp" = 1;

  };

  networking = {
    nat = {
      enable = true;
      externalInterface = "br0"; # Your external interface
      # Note
      # - for every routed network created in Terrraform, you need to add a new internal interface here
      # - and a static route needs to be added to the LAN router for the new network
      internalInterfaces = [
        "virbr1"
        "virbr2"
        "virbr3"
        "virbr4"
        "virbr5"
        "virbr6"
        "virbr7"
      ]; # Your KVM bridge interface
    };
    firewall = {
      enable = true;
      allowPing = true;
      allowedTCPPorts = [ ]; # Empty since we're allowing all traffic
      allowedUDPPorts = [ ]; # Empty since we're allowing all traffic
      extraCommands = lib.mkBefore ''
        # Allow all incoming and outgoing traffic on all interfaces
        iptables -A INPUT -j ACCEPT
        iptables -A OUTPUT -j ACCEPT
        iptables -A FORWARD -j ACCEPT
      '';
    };
  };
}
4

OK…

If others come here, it turns out the issue was not nixos. It was misleading because SSH was making it in, but HTTP was not. It turned out that my Cilium L2 Announcement Policy was incorrect for my Cilium Ingress on my Kubernetes cluster. I was looking “here” when I should have been looking “there.”