NixOS containers with macvlan, accessible from host

Help
1

Hello,

I have been experimenting with NixOS declarative containers and have some questions.

Goal: I want each container to appear on the local network with its own static IP address, and be accessible to every other container, as well as the host machine.

Current configuration.nix:

{ config, pkgs, lib, ... }:
{
  networking.defaultGateway = "192.168.0.1";
  networking.nameservers = [ ... ];
  networking.interfaces.enp3s0.ipv4.addresses = [
    { address = "192.168.0.151"; prefixLength = 24; }
  ];

  containers.test1 = {
    autoStart = true;
    ephemeral = true;
    macvlans = [ "enp3s0" ];

    config =
      { config, lib, pkgs, ... }:
      {
        networking.defaultGateway = "192.168.0.1";
        networking.nameservers = [ ... ];
        networking.interfaces.mv-enp3s0.ipv4.addresses = [
          { address = "192.168.0.152"; prefixLength = 24; }
        ];
      };
  };
  # repeat with containers.test2 and ip address 192.168.0.153, etc...
}

Additional information: The host interface is enp3s0, as stated in the above configuration.

Current state: Almost there. Each container is gets its own static ip address and is accessible on the network, as well as accessible from every other container. The only missing piece is that the host cannot access each container.

How should I modify my configuration.nix such that the host can reach each container?

ping @Mic92 who likely knows the answer :heart:

Thanks for any and all help.

1 Like
2

Hey! Is macvlan a requirement? I just accomplished this for a machine in a Makerspace I am part of, but without macvlan, instead using a br0 adapter. You can see how I’ve done it via the two commits below, which have their own verbose commit messages which explain what’s happening, hopefully it’s clear, if not feel free to ask for more information:
https://github.com/DoESLiverpool/nixcfgs/commit/ee17627ddea5a31528c339a7472757c747580ac9
https://github.com/DoESLiverpool/nixcfgs/commit/1c992535f28da21e46a2109e404fce95e9d9e3d0

If you want an example of using macvlan one can be found in the nixos tests nixpkgs/containers-macvlans.nix at 62f746b630d5189b2eedc19688ddcfb8021c1c71 · NixOS/nixpkgs · GitHub and it shouldn’t be too hard to extrapolate how it might apply to your own setup. Feel free to pester me in Matrix if you need some help hacking on these containers, and we can ultimately post a macvlan solution in this thread once we’ve figured it out.

2 Likes
3

Thanks @matthewcroughan, that seems to do the trick :tada:

1 Like
4

I tried to do something like this in the past, based on the macvlans example in the nixos tests. IIRC, it appeared to work, but then after some period of time the host mysteriously disappeared off the network and only the container could be contacted.

Keen to try out the br0 config when I get a chance! Is br0 limited to ethernet (as opposed to wireless) interfaces the way I believe macvlan is? One of the reasons I originally gave up was that I could only test them on the actual server with a wired connection, rather than messing about with containers on my (wireless) laptop then copying the config once I’d got them working. Having the server mysteriously disappear off the network was frustrating, given that it was only easily accessible via the network.

5

I know there may need to add some option, like

ip addr add brd