Networkd Sprint 2019-11-23/24 in Munich

As mentioned in the NixCon 2019 lightning talks, I want to discuss some ideas around the migration to networkd in NixOS. There are some new questions that have come up since my talk about networkd at NixCon 2018 and I think it would be wise to discuss them with more people from the community before starting to implement them.

We want to host a networkd sprint for everyone that is interested in discussing, implementing some ideas or just want to test networkd to possibly discover more issues. The ideal outcome would be an RFC for a redesign of the networking module for better composition and extensibility. We also have to decide what to do with scripted networking since we will most probably include a lot more of the features available in networkd. My goal would be to have networkd as a viable option for 20.03 and make it the default in at least 20.09 (including breaking compatibility for at least some of the networking options).

It will take place on a weekend at the new Mayflower office in Munich (Landsberger Strasse 314, OSM) on Saturday 2019-11-23 and the following Sunday. You can also already visit us on Friday. The next public transport station is Laim via the S-Bahn. Food and drinks will be provided for the duration of the sprint at least.

There will also be the possibility of joining remotely so you don’t necessarily need travel to Munich but that would of course simplify communication. If you’re unsure about accommodation, we can also host you in our office since we have a shower available (please contact us at nix@mayflower.de). There is a cheap hostel just a few 100 meters away (A&O Hostel) and relatively cheap hotels like the Super 8 Munich City West that are also close and in walking distance.

Dudle

Here is a poll to help decide on the date: https://dudle.inf.tu-dresden.de/nixos-networkd-sprint-2019/

Please try to vote as soon as possible so we can communicate the final date in about a week.

7 Likes

I think it’s to fix the date. According to the doodle, the most promising dates would be 2019-11-23 and 2019-12-14…

I was hoping for some more votes. :slight_smile: To break the tie I would choose 2019-11-23 since we have our company christmas party the day before 2019-12-14. I’ll edit the first post to that effect.

It would be awesome if anybody else who wants to attend physically apart from the dudle votes would either mention it here or contact us at nix@mayflower.de.

Just added my votes to the dudle before reading this, but I should be able to make the 23rd as well.

I was wondering if there is anywhere where the usecases you’ll be trying to support with this sprint are listed currently? Might I be able to provide some feedback on usecases for consideration in the design possibly if I don’t see something I’m currently doing or would like to do covered there?

I have what might be a somewhat unique setup to support a virtual domain across some VMs currently supported with NetworkManager and unbound. I’ve also struggled with the coffee shop/hotel wifi terms and conditions page use case.

I don’t think I’ll be able to attend virtually based on schedule so hoping to provide the feedback up front.

Thanks a lot for asking this. That would be very welcome since every use case would help us to design a networking module that just works for everyone without resorting to hacky bash scripting. If anybody else wants to chime in, please do!

It would be great if you and anybody else could sketch how you are using the networking module along the following questions.

  1. Are you working with different routing tables or networking namespaces or do you want to?
  2. Are you using networking.localCommands and for what?
    i.e. policy routing with ip rule, creating some special interface types, adding special routes, …
  3. If you’re using dhcp, do you have some special dhcpcd configuration?
  4. What’s your stance on predictable interface names? Do you like specifying hardware interfaces by their predictable interface name or would you rather match them in another way i.e. via mac address?
  5. Since systemd-resolved should also be introduced together with systemd-networkd, what dns resolving oddities are you dealing with and how you’re solving it currently?
    i.e. do you have an internal domain and need split dns via a vpn, that typical free wifi problem resolving problem if you have always-on-vpn
  6. Anything else you have additionally set up regarding networking in your NixOS configuration would be good to know, i.e. services that configure some aspect of the Linux networking stack that are not already part of the networking module

And some extra questions for some work after networkd but which we might need to consider for the networking module:

  1. Are you using the networking.firewall module? Since we know it’s quite limited, what features or options are you missing? You can think big here. :slight_smile:
  2. Are you using the networking.nat module and what are your thoughts on that?
3 Likes

Hotel booked. Looking forward :slight_smile:

This might not be a useful data point (apologies if not) but just in case it ends up being a useful:

I’ve used the networking module to force the traffic of some users through a VPN (and drop any traffic from those users if the VPN isn’t available).

It made pretty heavy use of the networking module (everything from making routing tables to maintaining iptables chains), you can find it here, hopefully there’s something in there you can draw inspiration for some improvements from.

1 Like

Not a frequent discourse reader and missed this dudle during NixCon. See you in Munich.

A bit to consider: How does networkd work together with other pieces of software which fuzz with the running network config and how does NixOS cover the whole situation:

  • VPN servers
  • BGP daemons
  • dockerd

I’m currently only using NixOS on my laptop and I think it would be nice if the following scenarios would be considered:

  • Hot plug support and automatic switching between wifi/cable (if this even needs consideration. I just remember having some problems in the past, but it might’ve been because of badly supported hardware)
  • The classic VPN routing problem with public networks
  • Rather simple Virtual Networks for development with kvm/virtualbox/etc.
  • The possibility to configure stuff very flexible and quickly. E.g.: I once had to debug a broken L3-Switch with a debian laptop. I added two virtual interfaces on one physical adapter and quickly rotated through various VLANS while staying connected to my office network through a vpn tunnel on another network interface. Having to rebuild the local config everytime I change the VLAN is not what I want, but I also still want to be connected to my office…
    I don’t know if this is already given or needs some consideration, but being able to keep existing (static) network configurations while temporarily tinkering around with iproute2 tools would be really nice :grin:

I also think that having one specific networking.firewall module for rather simple configurations is a great thing and I love it. It makes the setup of things like webservers very easy. :slight_smile:
It would be great if that module also works ‘‘out of the box’’ with other software like Docker, VPN servers, etc. (which I assume it doesn’t at moment?) but I imagine that could become a bit of a rabbit hole, as such software may add/remove network interfaces on their own…

1 Like

Here are some of my usecases for consideration:

  1. This first usecase is somewhat predicated on the assumption that systemd-resolved will be used as it seems somewhat linked to systemd-networkd but not totally required.
    I run a VM with a static IP which has a DNS server running a zone for other VMs. On nixos I currently have unbound configured with a private-domain and routes requests to the static IP of the bootstrap VM. Since NetworkManager has unbound set I think it configures resolvconf to write /etc/resolv.conf with nameserver 127.0.0.1. So requests are made to unbound and then it routes to the internal DNS server or a global one depending on the domain in question.
    This seems entirely possible with resolved without having an additional service as it seems to have some private domain features built in.

  2. I’m currently using nmcli to join various wifi networks. Adding the derived keys to the configuration didn’t seem like a great idea for static configuration as I think they’ll end up in the store and becomes world readable. Occasionally I have to join networks for a short period of time and then forget them. Mixed managed an non-managed config doesn’t seem to work but I could be wrong on that.

  3. On the note of short term wifi, hotel/coffee shop/airport networks that have a TOS page or some sort of web login, haven’t really worked well for me. It used to be that you could go to some page and you’d get redirected but with the level of use of https that can be difficult. On macOS and Windows there is some signal maybe during dhcp negotiation that says there is a page to authenticate with. I’m not totally sure how it works, but it would be great if that could open the link in your browser when you join such a network.
    Ideally the browser wouldn’t be using your user-profile but be more like a system web-view that doesn’t bring your cookies and plugins into the matter.

  4. Lastly joining a corporate Cisco VPN is something I’ve tried but haven’t gotten to work completely with Openconnect. I’ve been able to authenticate, but incorporating the appropriate DNS servers and routes is where I got stuck. I think there is a payload of network configuration delivered right after authentication which would need to update various config. This is where I think networkd can maybe make this work.

To answer your questions:

  1. I think I would want to in the openconnect case described above.
  2. I am not using networking.localCommands.
  3. I am using dhcp no special configuration but I did run into a bug there last week which I’m going to open a ticket for.
  4. I think I’m pro predictable interface names.
  5. Yes, internal domain for development on VMs, corporate domain via VPN, running in a coffee shop would be ideal but I don’t know if I can ask for all of that :slight_smile:.
  6. I run avahi for mDNS mostly for printer discovery.

I’m currently only running NixOS on my laptop at work, but I do some administration of a number of VMs on another distro. Certainly some features would make it easier to move to NixOS, such as running strongswan or an equivalent that can easily respond to re-keying events. I’m looking forward to using wireguard to encrypt traffic between VMs and ideally even authorized admins. This might be a little more than you intend to cover in the sprint but HA DNS Service discovery backed by a consensus protocol/service. (You said think big :slight_smile:)

1 Like

Are you working with different routing tables or networking namespaces or do you want to?

Yes, for my roadwarrior setup: https://www.wireguard.com/netns/#the-new-namespace-solution

I’d like to have a similar setup (maybe through the use of VRFs?) with networkd-wireguard.

Did you try https://nixos.org/nixos/options.html#programs.captive-browser ?

Thanks, I didn’t know about that, I’ll give it a try. I have some upcoming travel so this may come in handy.