Networkd Sprint 2019-11-23/24 in Munich

Development
10

A bit to consider: How does networkd work together with other pieces of software which fuzz with the running network config and how does NixOS cover the whole situation:

  • VPN servers
  • BGP daemons
  • dockerd
  • …
1 Like
11

I’m currently only using NixOS on my laptop and I think it would be nice if the following scenarios would be considered:

  • Hot plug support and automatic switching between wifi/cable (if this even needs consideration. I just remember having some problems in the past, but it might’ve been because of badly supported hardware)
  • The classic VPN routing problem with public networks
  • Rather simple Virtual Networks for development with kvm/virtualbox/etc.
  • The possibility to configure stuff very flexible and quickly. E.g.: I once had to debug a broken L3-Switch with a debian laptop. I added two virtual interfaces on one physical adapter and quickly rotated through various VLANS while staying connected to my office network through a vpn tunnel on another network interface. Having to rebuild the local config everytime I change the VLAN is not what I want, but I also still want to be connected to my office…
    I don’t know if this is already given or needs some consideration, but being able to keep existing (static) network configurations while temporarily tinkering around with iproute2 tools would be really nice :grin:

I also think that having one specific networking.firewall module for rather simple configurations is a great thing and I love it. It makes the setup of things like webservers very easy. :slight_smile:
It would be great if that module also works ‘‘out of the box’’ with other software like Docker, VPN servers, etc. (which I assume it doesn’t at moment?) but I imagine that could become a bit of a rabbit hole, as such software may add/remove network interfaces on their own…

1 Like
12

Here are some of my usecases for consideration:

  1. This first usecase is somewhat predicated on the assumption that systemd-resolved will be used as it seems somewhat linked to systemd-networkd but not totally required.
    I run a VM with a static IP which has a DNS server running a zone for other VMs. On nixos I currently have unbound configured with a private-domain and routes requests to the static IP of the bootstrap VM. Since NetworkManager has unbound set I think it configures resolvconf to write /etc/resolv.conf with nameserver 127.0.0.1. So requests are made to unbound and then it routes to the internal DNS server or a global one depending on the domain in question.
    This seems entirely possible with resolved without having an additional service as it seems to have some private domain features built in.

  2. I’m currently using nmcli to join various wifi networks. Adding the derived keys to the configuration didn’t seem like a great idea for static configuration as I think they’ll end up in the store and becomes world readable. Occasionally I have to join networks for a short period of time and then forget them. Mixed managed an non-managed config doesn’t seem to work but I could be wrong on that.

  3. On the note of short term wifi, hotel/coffee shop/airport networks that have a TOS page or some sort of web login, haven’t really worked well for me. It used to be that you could go to some page and you’d get redirected but with the level of use of https that can be difficult. On macOS and Windows there is some signal maybe during dhcp negotiation that says there is a page to authenticate with. I’m not totally sure how it works, but it would be great if that could open the link in your browser when you join such a network.
    Ideally the browser wouldn’t be using your user-profile but be more like a system web-view that doesn’t bring your cookies and plugins into the matter.

  4. Lastly joining a corporate Cisco VPN is something I’ve tried but haven’t gotten to work completely with Openconnect. I’ve been able to authenticate, but incorporating the appropriate DNS servers and routes is where I got stuck. I think there is a payload of network configuration delivered right after authentication which would need to update various config. This is where I think networkd can maybe make this work.

To answer your questions:

  1. I think I would want to in the openconnect case described above.
  2. I am not using networking.localCommands.
  3. I am using dhcp no special configuration but I did run into a bug there last week which I’m going to open a ticket for.
  4. I think I’m pro predictable interface names.
  5. Yes, internal domain for development on VMs, corporate domain via VPN, running in a coffee shop would be ideal but I don’t know if I can ask for all of that :slight_smile:.
  6. I run avahi for mDNS mostly for printer discovery.

I’m currently only running NixOS on my laptop at work, but I do some administration of a number of VMs on another distro. Certainly some features would make it easier to move to NixOS, such as running strongswan or an equivalent that can easily respond to re-keying events. I’m looking forward to using wireguard to encrypt traffic between VMs and ideally even authorized admins. This might be a little more than you intend to cover in the sprint but HA DNS Service discovery backed by a consensus protocol/service. (You said think big :slight_smile:)

2 Likes
13

Are you working with different routing tables or networking namespaces or do you want to?

Yes, for my roadwarrior setup: Routing & Network Namespaces - WireGuard

I’d like to have a similar setup (maybe through the use of VRFs?) with networkd-wireguard.

2 Likes
14

Did you try NixOS Search ?

15

Thanks, I didn’t know about that, I’ll give it a try. I have some upcoming travel so this may come in handy.

16

Train tickets booked, will crash at the office, looking forward to see you all there!

1 Like
17

I am not using mainline NixOS, so feel free to disregard, but I do have some usecase observations (these are all things that have happenned to me this year).

  1. Of course no-rebuild on-the-fly reconfiguration.

  2. There are networks where you want to use IPv6, and there are networks where you really don’t; what are the best ways to configure it?

  3. There are cases where I need to use ethernet connection for one subnet, but WiFi is actually better for most of the external subnets, so I want to have both cable and WiFi on with some simple routing rules.

2a. (and of course some of the traffic should be routed through a VPN)

  1. Some networks have a rogue DHCP server that would be really nice to blacklist.

  2. Sometimes running your own recursive resolver is actually a better idea than using the network-provided one, sometimes no; ideally one could use a local resolver for a local zone and decide per-network whether to forward most of the requests to the DHCP-provided DNS resolver.

I would be interested in participating to do some requirements work, maybe first drafts of documentation, and to learn to do Python NixOS tests (whatever comes out of the sprint, it needs some NixOS tests, right?). I will definitely not test anything systemd on my laptop. and probably not write implementation code. Am I too far out of the target audience, or could I be useful as a test rabbit who knows basic stuff about networks but nothing about networkd?

1 Like
18

@fpletz Doodle (the link is now hidden into a spoiler) seems to imply that the get-together time on Saturday starts at 10:00; should this be a part of the visible text (probably with a quick confirmation message so that people get notifications)?

1 Like
19

Start at 10 was also my impression.

20

As I won’t be ale to attend remotely, I wanted to share a few answers and thoughts :

  1. Yes, I use different routing tables, but not namespaces.
  2. I configure everything through systemd.network. The only exception is that I use localCommands to IPv6 null-route cache.nixos.org, on every machine.
  3. When I use DHCP, it’s mostly through Network-Manager.
  4. I use predictable names on physical hosts, but tend to to match on MAC address in VMs, so it depends.
  5. I often have my own resolver, so I disable resolved, and use dnsdist to handle split DNS. Avoiding VPN leaks is done using routing tables and firewall marks (using namespaces could be better).
  6. On my personal machines, networking has usually been enough, except when I had to setup point-to-point links. On servers with complex networking needs, networking is not enough and I have to use systemd.network directly. networkd seems to fully cover the Linux networking API, while there are lots of things missing in networking.

Extra:

  1. I only use networking.firewall in application containers and on my personal machines, it’s too cumbersome to use to write complex forwarding firewalls, and in that case I use ferm.
  2. I never used it, as on the single machine where I have to do NAT, I’m using multiple external IPs, and want to NAT networks to specific IPs.

Additional answers:

  1. I’m using BIRD2 to handle OSPF. It seems to work fine with NixOS and the current networking things.
  2. I use Wireguard tunnels, but I don’t use the NixOS module for this, I configure it as a netdev in networkd, because I need to add rules and routes to specific tables. IIRC, networking.wireguard allows configuring the routing tables for routes generated by Wireguard, but does not allow configuring additional matching rules.
  3. When using systemd.network, I’ve had a few times to resort to extraConfig because of missing options (especially for routing policy rules).
  4. I still use networking for a few things on the servers on which I use systemd.network : enabling Wireguard (just having the wireguard interfaces in systemd.network is not enough), configuring extra hosts in /etc/hosts, and adding iproute2 table aliases.
  5. I used systemd.network because I really wanted to avoid using custom bash scripts full of ip and iptables invocations.

Also:

  • I think switching to networkd is a good idea for NixOS, even if it may come at some cost. From my experience, networkd allows good access to Linux’s networking API, but suffers sometimes of some opinionated choices. It would have huge benefits, as the current networking module is somewhat limited. I’m very glad that networkd can already be configured directly through NixOS, I like the way networkd is structured (separating netdevs, interfaces and networks) and think it could serve as a good basis for a reworked networking module. One of my few gripes with networkd is that it’s sometimes way too much verbose, and I don’t how much can it be made to support on-the-fly temporary reconfiguration, which seems to be a common requirement.
2 Likes
21

Sorry, I should’ve posted that earlier. I will update the post above, too. We will open the office at around 10:00 CET and have some breakfast available. We will also already start the video conference and see how it goes.

Since we have people arriving at around 11:00 CET, the first session/planning will begin at 12:00 CET / 11:00 UTC.

1 Like
22

I’m planning to join you in Munich tomorrow and on Sunday.

23

We will have this video conference open at all times if anybody wants to join.

(Sorry about the company plug in the room name, we have a paid whereby account with fixed channel names. The alternatives we tried didn’t work as good as whereby.)

24

We’re currently discussing & hacking. Remote participants still welcome.

I’d like to sum up some points we’ve come up with so far:

  • We will have two networking backends: the old scripted and the new networkd-based one. At least for the following release, both of them will be supported. In 20.09, we will switch the default.
  • The networking.* name space should be cleaned up so that it contains only options for core networking. Stuff on top like connman, networkmanager, or ssmtp should be moved to services.* or services.networking.*.
  • What remains in networking.* should be implemented by both backends.
  • Each backend may define specific options which allow to make use of the individual strengths.

@fpletz, @flokli and @andir are currently figuring out what the new layout could look like. They will present their findings to the broader community.

8 Likes
25

Friendly ping.

Has this been published somewhere?

3 Likes
26

There was this pad being shared during the sprint - not sure how much of it was done as I wasn’t present.

Looking forward to a report! :wink:

1 Like
27

Has there been any progress on these points?

28

@asymmetric I’ve not been directly involved. @fpletz / @flokli / @andir any news on this?

29

I tried to do a writeup with @fpletz some moths ago, but apparently we were both stuck into many different other things. I’ll try to get back to him and get this report published. Sorry for the radio silence.