Docker hanging on reboot

Help
1

Anybody else having a problem with Docker hanging if containers are still running at reboot?

I tried this solution but it isn’t working …

https://github.com/containerd/containerd/issues/5502#issuecomment-1019937241

2

It’s likely this is caused by docker running a process that ignores the usual SIGTERM. You’ll need to ask docker to use SIGKILL for that process, or fix the process in question.

Rather than writing another systemd unit that mercilessly murders all your containers, I’d suggest adding an ExecStop to the offending unit that asks docker to specifically SIGKILL that container with docker kill -s SIGKILL.

3

Thanks @TLATER ! That makes sense. Is there a good way to tell which containers are the offending ones? I don’t see anything helpful in the console output while the machine is rebooting. Is almost seems like every container on my machine is a problem.

Also how do I add ExecStop to an offending unit? I am just using docker-compose to start and stop these containers.

4

Ah, that’s annoying. I’d assumed you were launching docker containers with systemd.

A quick way to check if any of them have this problem is to use docker-compose kill -s SIGTERM on each. If any of them are hanging that should take longer than usual.

Alas, not option. Writing a unit to SIGKILL a specific container (whichever ones end up hanging) is probably the best option in this case. I’m not sure if docker-compose has some strange interaction with how containers are killed on shutdown, so maybe my theory is wrong.

5

For me using virtualisation.docker.liveRestore = false helped to mitigate the “waiting for a stop job docker” thing.

3 Likes
6

Thanks @NobbZ that did the trick!

7

virtualisation.docker.liveRestore = false did the trick for me too.

But why is virtualisation.docker.liveRestore = true the default? Vanilla docker defaults this to false: Live restore | Docker Docs. Also

This option is incompatible with docker swarm.

If I don’t hear anything for a couple of days, I’ll file a bug (or PR) for this.

1 Like
8

Hi, I have been suffering the issue or similar one for some time, and I have found what was cause, at least for me.
You might find it useful as well.
I posted it here