Containers dynamic network config

Help
1

Hi,

I’m trying to create containers on my system without specifying IP addresses for them. The containers should be configured using DHCP. I saw a talk about this, but without any clear details on how to accomplish this. In this talk, the presenter also mentioned that the containers are accessible via their hostname. I got that part working, but I can’t figure out how to correctly set up the DHCP part. What I’ve got now (not working):

on my host

{ lib, ... }: {
  networking = {
    useNetworkd = true;
    useDHCP = false;
    domain = "vpn.sustainablemotion.io";
    firewall = { enable = true; };
    nat = {
      enable = true;
      internalInterfaces = [ "ve-*" ];
      externalInterface = "br0";
    };
  };
  services.resolved = {
    domains = [ "vpn.sustainablemotion.io" "sustainablemotion.io" ];
    fallbackDns = [ "9.9.9.9" "149.112.112.112" ];
  };

  systemd.network = {
    enable = true;
    netdevs = {
      "20-br0" = {
        netdevConfig = {
          Kind = "bridge";
          Name = "br0";
        };
      };
    };
    networks = {
      "10-uplink" = {
        matchConfig.Name = lib.mkDefault "enp1s0";
        networkConfig = {
          DHCP = "ipv4";
          # IPv6AcceptRA = true;
          Bridge = "br0";
        };
        dhcpV4Config = { UseDNS = false; };
        linkConfig.RequiredForOnline = "routable";
      };
      "20-br0" = {
        matchConfig.Name = "br0";
        networkConfig = { DHCPServer = true; };
        dhcpServerConfig = { ServerAddress = "172.16.0.1/12"; };
        linkConfig = { RequiredForOnline = "routable"; };
      };
      "40-vb" = {
        matchConfig.Name = "vb-*";
        networkConfig = { DHCP = "ipv4"; };
      };
    };
  };
}

and the container:

...
        privateNetwork = true;
        hostBridge = "br0";

        config = {          
          networking={
            interfaces."eth0".useDHCP = true;
            useHostResolvConf = false;
            firewall = {
              enable = true;
              allowedTCPPorts = [ 5432 ];
            };
          };
          services.resolved.enable = true;
...

Does anyone have a working example or can anyone point me in the right direction?

2

Maybe I’m way off the mark, I haven’t tried this in a container. In any case, I used nm2nix script to configure the network interface via .nix file. Run nmtui in the container to configure the interface, then run nm2nix as per the github instructions. → GitHub - Janik-Haag/nm2nix: Converts .nmconnection files into nix code

3

I now have the bridge network device and a veth device for the container. I see that the DHCP server is running on the bridge device and the veth device is requesting an address via DHCP but never gets one. I don’t understand why this is not working?

$ networkctl
IDX LINK            TYPE      OPERATIONAL SETUP
  1 lo              loopback  carrier     unmanaged
  4 enp8s0f3u2u3c2  ether     off         unmanaged
  5 enp195s0        ether     routable    configured
  9 br0             bridge    routable    configured
 13 vb-postgrespm8F ether     enslaved    configuring

with:

$ networkctl status br0
● 9: br0
                   Link File: /nix/store/w9qcpyhjrxsqrps91wkz8r4mqvg9zrxc-systemd-256.10/lib/systemd/network/99-default.link
                Network File: /etc/systemd/network/20-br0.network
                       State: routable (configured)
                Online state: online
                        Type: bridge
                        Kind: bridge
                      Driver: bridge
            Hardware Address: c6:88:8f:2e:bd:95
                         MTU: 1500 (min: 68, max: 65535)
                       QDisc: noqueue
IPv6 Address Generation Mode: eui64
               Forward Delay: 15s
                  Hello Time: 2s
                     Max Age: 20s
                 Ageing Time: 5min
                    Priority: 32768
                         STP: no
      Multicast IGMP Version: 2
                        Cost: 2000
                  Port State: disabled
    Number of Queues (Tx/Rx): 1/1
            Auto negotiation: no
                       Speed: 10Gbps
                     Address: 172.16.0.1
                              fe80::c488:8fff:fe2e:bd95
           Activation Policy: up
         Required For Online: yes
         Offered DHCP leases: none

and

$ networkctl status vb-postgrespm8F
● 13: vb-postgrespm8F
                   Link File: /nix/store/w9qcpyhjrxsqrps91wkz8r4mqvg9zrxc-systemd-256.10/lib/systemd/network/80-container-vb.link
                Network File: /etc/systemd/network/30-container.network
                       State: enslaved (configuring)
                Online state: offline
                        Type: ether
                        Kind: veth
                      Driver: veth
           Alternative Names: vb-postgresql-17
            Hardware Address: 52:7c:2f:b3:98:6b
                         MTU: 1500 (min: 68, max: 65535)
                       QDisc: noqueue
                      Master: br0
IPv6 Address Generation Mode: none
    Number of Queues (Tx/Rx): 64/64
            Auto negotiation: no
                       Speed: 10Gbps
                      Duplex: full
                        Port: tp
           Activation Policy: up
         Required For Online: yes

the network files:

$ cat /etc/systemd/network/20-br0.network
[Match]
Name=br0

[Network]
DHCPServer=true
IPMasquerade=ipv4
IPv4Forwarding=true
IPv6AcceptRA=false
IPv6SendRA=false
Address=172.16.0.1/12

[DHCPServer]
ServerAddress=172.16.0.1/12

and:

$ cat /etc/systemd/network/30-container.network
[Match]
Kind=veth
Name=vb-*

[Link]
RequiredForOnline=routable

[Network]
Bridge=br0
DHCP=ipv4

and some logs:

Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: loaded "bridge"
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Using MAC address: c6:88:8f:2e:bd:95
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Creating
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: netdev has index 9
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: netdev ready
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Saved new link: ifindex=9, iftype=ETHER(1), kind=bridge
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Saved original MTU 1500 (min: 68, max: 65535)
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Saved hardware address: c6:88:8f:2e:bd:95
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Saved IPv6 link-local address generation mode: eui64
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Flags change: +UP +LOWER_UP +RUNNING +MULTICAST +BROADCAST
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Link UP
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Gained carrier
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: udev initialized link
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Saved new link: ifindex=13, iftype=ETHER(1), kind=veth
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Saved original MTU 1500 (min: 68, max: 65535)
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Saved hardware address: 52:7c:2f:b3:98:6b
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Attached to master interface: 9
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Saved IPv6 link-local address generation mode: none
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Flags change: +UP +LOWER_UP +RUNNING +MULTICAST +BROADCAST
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Link UP
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Gained carrier
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: udev initialized link
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Found driver: veth
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Saved permanent hardware address: 00:00:00:00:00:00
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Received new foreign QDisc (configured): handle=0:0, parent=ffff:ffff, kind=noqueue
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Received new foreign QDisc (configured): handle=0:0, parent=ffff:ffff, kind=noqueue
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Received new foreign address (configured): 172.16.0.1/12 broadcast 172.31.255.255 (valid forever, preferred forever), flags: permanent, scope: global, label: n/a
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is in pending state.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Received new foreign address (configured): fe80::c488:8fff:fe2e:bd95/64 (valid forever, preferred forever), flags: permanent, scope: link
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Gained IPv6LL
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is in pending state.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Received new foreign route (configured): dst: 172.16.0.0/12, src: n/a, gw: n/a, prefsrc: 172.16.0.1, table: main(254), priority: 0, proto: kernel, scope: link, type: unicast, flags: n/a
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Received new foreign route (configured): dst: 172.16.0.1/32, src: n/a, gw: n/a, prefsrc: 172.16.0.1, table: local(255), priority: 0, proto: kernel, scope: host, type: local, flags: n/a
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Received new foreign route (configured): dst: 172.31.255.255/32, src: n/a, gw: n/a, prefsrc: 172.16.0.1, table: local(255), priority: 0, proto: kernel, scope: link, type: broadcast, flags: n/a
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Received new foreign route (configured): dst: fe80::c488:8fff:fe2e:bd95/128, src: n/a, gw: n/a, prefsrc: n/a, table: local(255), priority: 0, proto: kernel, scope: global, type: local, flags: n/a
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: netdev exists, using existing without changing its parameters
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Created
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Found driver: bridge
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Saved permanent hardware address: 00:00:00:00:00:00
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Bridge parameters set success
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Link state is up-to-date
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: State changed: pending -> initialized
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: found matching network '/etc/systemd/network/20-br0.network'.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Configuring with /etc/systemd/network/20-br0.network.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: State changed: initialized -> configuring
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv6/conf/br0/disable_ipv6' to '0'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv4/conf/br0/forwarding' to '1'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv6/conf/br0/use_tempaddr' to '0'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv6/conf/br0/accept_ra' to '0'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv6/conf/br0/proxy_ndp' to '0'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv4/conf/br0/promote_secondaries' to '1'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Requested to activate link
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Requesting DHCP server.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Requesting static address (configured): 172.16.0.1/12 (valid forever, preferred forever), flags: n/a, scope: global, label: n/a
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Setting addresses
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Bringing link up
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Link state is up-to-date
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: State changed: pending -> initialized
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: found matching network '/etc/systemd/network/30-container.network'.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Configuring with /etc/systemd/network/30-container.network.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: State changed: initialized -> configuring
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv4/conf/vb-postgrespm8F/forwarding' to '1'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv6/conf/vb-postgrespm8F/use_tempaddr' to '0'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv6/conf/vb-postgrespm8F/accept_ra' to '0'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv6/conf/vb-postgrespm8F/proxy_ndp' to '0'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: Setting '/proc/sys/net/ipv4/conf/vb-postgrespm8F/promote_secondaries' to '1'
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Requested to set master interface
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Requested to set bridge configurations
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Requested to activate link
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Requested configuring of the DHCPv4 client.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link layer is configuring.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: LLDP Rx: Started LLDP client
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Setting bridge configurations
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): static addresses are not configured.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: bridge configurations set.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): link is not activated.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: Bringing link up
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Configuring static address (requesting,configured): 172.16.0.1/12 broadcast 172.31.255.255 (valid forever, preferred forever), flags: n/a, scope: global, label: n/a
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: link_check_ready(): dynamic addressing protocols are enabled but none of them finished yet.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Received updated static address (configured): 172.16.0.1/12 broadcast 172.31.255.255 (valid forever, preferred forever), flags: permanent, scope: global, label: n/a
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: link_check_ready(): static addresses are not configured.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Addresses set
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: State changed: configuring -> configured
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Configuring DHCP Server.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Not emitting NTP servers on link, couldn't find suitable uplink.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Not emitting SIP servers on link, couldn't find suitable uplink.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: DHCPv4 server: STARTED
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: br0: Offering DHCPv4 leases
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client: STARTED on ifindex 13
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client is configured, acquiring DHCPv4 lease.
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client: DISCOVER
Apr 02 13:53:58 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client: State changed: initialization -> selecting
Apr 02 13:54:02 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client: DISCOVER
Apr 02 13:54:05 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client: DISCOVER
Apr 02 13:54:14 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client: DISCOVER
Apr 02 13:54:31 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client: DISCOVER
Apr 02 13:55:02 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client: DISCOVER
Apr 02 13:56:06 hetzner-prod01 systemd-networkd[343861]: vb-postgrespm8F: DHCPv4 client: DISCOVER