NixOS + Docker Swarm networking fails to start

Help
1

I can’t seem to figure out how to deploy a docker stack to NixOs on a VPS. The service fails to start with an error:

# docker service ps <service_name> --no-trunc
network sandbox join failed: subnet sandbox join failed for "10.0.0.0/24": error creating vxlan interface: operation not supported

After some time the error becomes:

network sandbox join failed: network sandbox join failed: could not get network sandbox: unshare: no space left on device

Docker-related nix config:

virtualisation.docker.enable = true;
virtualisation.docker.rootless.enable = false;

Container is a simple web app that uses a port binding.

2

I run multiple docker swarm instances on NixOS but I never got this kind of error.

Its seems related to your network config, on my side I only used network manager or static interfaces with networking.interfaces. Maybe the IP you use when you run docker swarm init --advertise-addr <IP> has an impact.

3

I’m a beginner with both NixOS and Docker so I might be missing something obvious.

This is a new VPS with a NixOS 24.11 install. Could I break down the steps somehow to know what exactly fails? Do I need to add any specific networking configuration?

Docker Registry seems to be running fine at the same time:

services.dockerRegistry.enabled = true

I’ve tried

docker swarm leave -f
docker prune -a
docker swarm init # no advertise-addr

No dice.

4

No I haven’t done any specific network config for swarm, but it could be useful to share your network configuration.

When I init swarm I always specify the advertise-addr and the default address pool, you can try something like this:

docker swarm init --advertise-addr YOUR-PUBLIC-IPV4-ADDR --default-addr-pool 172.16.0.0/12 --default-addr-pool-mask-length 24

My docker config has nothing special:

virtualisation.docker = {
  enable = true;
  autoPrune.enable = true;
  autoPrune.flags = [ "-af" "--filter='label!=avoid-prune'" ]; # this label filter is for a special use case
  liveRestore = false; # this should be put for swarm but I don't think it's related to your issue
};
1 Like