But the actual mac address in the container is always d6:24:a0:56:8a:3e (on my machine anyway).
[root@x:~]# ip link show eth0
2: eth0@if19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether d6:24:a0:56:8a:3e brd ff:ff:ff:ff:ff:ff link-netnsid 0
Do dmesg or the unit logs in the container say anything useful?
My guess is that, since this creates a network using the physical card of the host before the container is involved, the MAC address cannot actually be changed in the container without some higher permissions. systemd-nspawn doesn’t seem to have any settings for doing so either, so you may be stuck.
Is there any reason for why you’d like to change it?
dmesg is empty, and journalctl doesn’t show how the mac address got assigned.
I need to have deterministic (unchanging) IP addresses assigned to various machines, containers and VMs that are participating directly on the LAN (via bridge in the case of virtual), even after a rebuild (so that the firewall can forward to the right machines). The only way I can do this without manually configuring the IP stack on each and every one separately is to have unchanging MAC addresses on them so that the DHCP server’s static lease table can reliably assign addresses.
I was hoping that networking.interfaces.eth0.macAddress would do the trick, but it seems to be buggy. Although I’m not sure why the systemd link is ignored… I have no problem setting the mac address in LXD containers and KVM in Ubuntu.
dmesg should never be empty, I did mean the host side It shouldn’t have anything useful though, I don’t believe the settings you set ever make it to the kernel.
Since you’re comparing this to KVM, I guess you might need a refresher on what is involved here? Quick summary, but you can probably find better details online (caveat, I’m not interested in and haven’t used LXD containers, so my knowledge may be off):
KVM: uses the virtualization features of your CPU to launch a whole operating system, kernel and all, in its own address space, and puts virtual devices in place for everything non-software. These virtual devices are designed to be used by the kernel as if they were real devices, allowing the guest kernel to modify them as it pleases without interfering with host hardware. It’s an entirely different approach to the other two, and naturally allows more fine-grained control.
systemd-nspawn (underlying NixOS containers): uses kernel namespaces, which crucially are not virtualization technology. This is simply the kernel providing certain features in a namespaced way, so that processes cannot overlap them; everything still runs in the same address space, and with the same kernel. Systemd-nspawn is designed to spin up simple containers for systemd units, so that those can be isolated and secured better, allowing full use of kernel features. The containers arising from this are a nice side effect, but systemd-nspawn simplifies a lot of the configuration for ease of use.
LXD containers: use kernel namespaces to run the processes; unlike systemd-nspawn, however, it is designed to mimic virtual machines like the KVM ones. It has more complex network configuration, and allows more fine grained control than systemd-nspawn, at a cost of more complex configuration that is usually not necessary in the context of simple units.
So, in essence, while all three use similar things to provide the virtual device, systemd-nspawn is simply not designed to give too much freedom in defining it.
Setting the MAC address in the container probably doesn’t work because the process running your networking service doesn’t have the requisite permissions to change the interface on the host, but I’m not sure; you’d need to check the systemd-nspawn source to know. LXD containers probably give you the correct namespaces to allow that (while trading off some security to do so), and KVM anyway allows the process to do almost anything, since it’s not the host kernel.
Nonetheless, your use case should be achievable. The MAC address is generated by systemd based on the host and guest names. This blog goes through some of the basics, and at the end links a script with which the MAC addresses can be predicted.
Note also that both LXD and KVM are available in nixos, as well as docker and podman, so if nixos-containers aren’t flexible enough there are always options.
# cat container-x.nix
{ config, pkgs, ... }:
{
containers.x = {
autoStart = true;
ephemeral = true;
# Use a specific bridge instead of the host's networking
privateNetwork = true;
hostBridge = "br0";
config = { config, pkgs, ... }: {
boot.isContainer = true;
networking.interfaces.eth0.useDHCP = true;
# Hack: Change the MAC address before dhcpcd runs
systemd.services.setmacaddr = {
script = ''
/run/current-system/sw/bin/ip link set dev eth0 address 02:12:34:56:78:90
/run/current-system/sw/bin/ip addr flush eth0
'';
wantedBy = [ "basic.target" ];
before = [ "dhcpcd.service" ];
};
};
};
}
It’s using a bridge (br0) that the host is bridging to the physical LAN like so:
imports = [
./container-x.nix
];
networking = {
bridges.br0.interfaces = [ "enp1s0" ];
useDHCP = false;
interfaces.enp1s0.useDHCP = false;
interfaces.br0 = {
useDHCP = true;
# We're on a physical machine so this works fine
macAddress = "02:12:34:56:78:00";
};
};
UPDATE: It seems that systemd is a little flaky depending on what other services you’re running. I’ve found that some of my containers will run my hacked service AFTER dhcpcd despite my telling it to run before. If that happens, you can do a hack-upon-a-hack to make it work:
# Hack: Change the MAC address after dhcpcd runs
systemd.services.setmacaddr = {
script = ''
/run/current-system/sw/bin/ip link set dev eth0 address ${macaddr}
/run/current-system/sw/bin/systemctl stop dhcpcd.service
/run/current-system/sw/bin/ip addr flush eth0
/run/current-system/sw/bin/systemctl start dhcpcd.service
'';
wantedBy = [ "basic.target" ];
after = [ "dhcpcd.service" ];
};
Basically, tell it to run after dhcpcd, then manually shut down dhcpcd, flush eth0, then restart dhcpcd.
Yeah.
Anyway, ugly as hell and it ends up requesting an extra address under the old MAC address from your DHCP server (once only on container start), but at least it’s less flaky.
Static IP addresses would technically work, but it’s a LOT less painful if I can ignore the nightmare of all the ideosyncratic ways of configuring IP addresses on all the various boxes (different OS, different stacks, different virtualization, etc) and just control everything centrally from a DHCP server and a firewall server. That’s what DHCP was invented to solve (under the assumption that MAC addresses rarely if ever change, of course).
It shouldn’t have anything useful though, I don’t believe the settings you set ever make it to the kernel.