Networking: need to configure networking a bit differently

Help
1

Hi everyone,

For my colocated server I want to configure networking slightly differently from what NixOS currently offers.

I have written a detailed bug report over at GitHub:

Can someone please help me with this?

Thanks, Mark

2

So either I figure out how to change the module and use that for my own flake, and submit a PR once everything works as I’d like, or, I’d disable networking altogether and use a custom networking systemd init file. I’m at a loss for either. So pointers will be very much appreciated.

3

I haven’t read through your full list of suggestions (nor do I think I can implement them without thinking really hard about your use cases), but the implementation of networking.interfaces lives here: nixpkgs/network-interfaces-scripted.nix at 2b1bba76a13ed39c7abc0a6e8f74f9e168cf3c7c · NixOS/nixpkgs · GitHub

It’s just a shell script, and a pretty straightforward one at that. Shouldn’t be too hard to add what you need. Some of the things you suggest are even already possible, I think.

You don’t have to modify the upstream module to do these things though. The module allows you to run arbitrary commands, so anything networking related you want to do is possible on NixOS: networking.localCommands.

Furthermore, if you’re really stuck with something bespoke that isn’t possible through the networking module yet, even with arbitrary shell commands (?), you can fall back to the systemd-networkd implementation: systemd.network.

It’s quite thorough in mapping pretty much all networking features of the kernel, so you should have no problems implementing practically any networking config with it. IME using it is also a bit easier because you can actually use other forums/search engines to get help with systemd-networkd, unlike the NixOS networking module.

Aiui, though I might misremember, NixOS 23.05 will also migrate the current script-based networking implementation to the systemd-networkd based one.

2 Likes
4

I guess my bug report was a bit too long. I had an episode of late-night bugreporteritis. :sweat_smile: thanks for responding.

I don’t know how to test a modified version of this file in my local flake config. I’m using @Misterio’s nix-starter-config (standard). Mistorio, maybe you can let me know how to do this?

One thing that imo does have to be added here, is that the broadcast address has to be properly set. As in adding brd + before dev on line 207 (and 210) will do that.

Secondly: I want to be able to postfix preferred_lft 0 to any (secondary) IP address when it gets added.

Third - I don’t know how to fix this: before an IPv6 route is added, the script must wait for dad (duplicate address detection) to finish before adding a route. Workaround: disable dad. Better: wait until the address does not show “tentative” anymore as described in this systemd bug report.

I can also test systemd.network. Haven’t looked into that yet. Will try. Thanks for that tip (and the localCommands one).

5

You can substitute your own copy of the module: NixOS 22.11 manual | Nix & NixOS

I.e., just copy the file from upstream, modify it, sub it in using that. Or do something nicer with a git clone :wink:

1 Like
6

Thank you! And thanks for being patient with me.

I was just reading up on the Wiki about systemd-networkd. I’ll go take that route first. It looks like I can set preferred_lft to 0 (systemd networking supports it anyway).

And also, since you’ve hinted at 23.05 moving to this networking setup, it’s probably better to try that out first than to shoehorn in my changes to something that will become deprecated sooner or later.

I’ll give a status-update on my progress in this thread once I’ve configured it properly.

1 Like
7

Alright, it seems like systemd-networkd kinda does what I need, in that I’m able to setup networking properly! It makes no sense to do this with networking.* if we’re going to move to systemd based networking. The systemd config is somewhat convoluted, but works fine. A little example:

    "20-lan" = {
        matchConfig.Name = "eth1";
        DHCP = "no";
        networkConfiguration = {
            IPv4ProxyARP = true;
            IPv6ProxyNDP = true;
            IPv6ProxyNDPAddress = [
                "2001:db8:100:1001::2",
                "2001:db8:100:1002::2"
            ];
        };
        address = [
            "192.0.2.1/24",
            "2001:db8:100:1000::1/60"
        ];
        routes = [
            { routeConfig = { Destination = "192.0.3.1/24"; PreferredSource = "192.0.2.1"; }; }
            { routeConfig = { Destination = "2001:db8:100:1001::/60"; PreferredSource = "2001:db8:100:1000:1"; }; }
        ];
        linkConfig = {
            RequiredForOnline = "routable";
            MTUBytes = "9000";
        };
    };

I had to add these lines to ensure that openssh starts properly afterwards:

systemd.services.sshd.wants = [ "network-online.target" ];
systemd.services.sshd.after = [ "network-online.target" ];

Much thanks for the help. I can close my bug report since it’s no longer needed.

1 Like
8

Note that the migration isn’t really intended to switch to systemd.network as you use it, it’s just that the networking.* options will be converted to systemd.network ones instead of the shell scripts that currently exist. networking.* does have its place, it’s an abstraction that makes things:

  1. Super easy plug-and-play for people who don’t know their networking that well (such as yours truly).
  2. Agnostic to future changes in how Linux network management is done; just in case we migrate away from systemd-networkd again one day.

That said, the NixOS networking module can’t realistically compete with the current level of maintenance of systemd-networkd. It will probably be a more usable interface to network configuration for the foreseeable feature, it’s just a bit more verbose. Once you have an even slightly advanced use case I don’t really see a reason not to use systemd-networkd directly.

1 Like