Overriding/modifying systemd unit file

I am using this on NixOS to run docker containers:

virtualisation = {
    containers.enable = true;
    oci-containers.backend = "docker";
    docker = {
      enable = true;
    };
};

but I need to modify the systemd unit file to include

[Mount]
LogLevelMax=0

What I would have expected:

  systemd.services.docker = {
    overrides = ''
      [Mount]
      LogLevelMax=0
    '';
  };

Any pointers to get this working?

The way to do this on a service is with

systemd.services.docker.serviceConfig.LogLevelMax = 0;

nixos will handle turning that into a proper drop-in unit file.

But I’m guessing this isn’t really what you want. If you just want to stop all the run-docker-*.mount: Succeeded messages, this is going to be trickier. It’s not the docker unit you want to silence; it’s the mount units it’s causing by mounting stuff. That StackOverflow answer isn’t a full answer, because it’s not telling you that you have to create a drop-in unit like that for every one of these mounts. Not for the docker service. For every mount that you want to silence.

NixOS also makes this a little trickier because the systemd.mounts.* option is a list, the unit file name is derived from the where option, and the what option isn’t optional. Apparently whoever chose this design didn’t consider the possibility of wanting to make drop-ins for transient mount units… So while you can’t get the nice api we’d use for other unit types, you can use the lower level api to do it with raw unit text:

  systemd.units."foo.mount" = {
    overrideStrategy = "asDropin";
    text = ''                                                                                                                                                                                 
      [Mount]                                                                                                                                                                                 
      LogLevelMax=0                                                                                                                                                                           
    '';
  };

and again, you have to do this for every mount you want to silence.

1 Like

:flushed: Uff

What creates all those the mount units?
Why couldn’t it be set there?

And why are they happening so often? The containers are not restarting.

Docker is doing its own mounts. It’s doing the equivalent of mount foo /bar. Systemd turns every single mountpoint created by other things into a transient mount unit. It’s not systemd creating these, so systemd doesn’t have a way to apply mount properties to all of them.

1 Like

Got it. But why would docker execute these mounts so often - not just on container startup?

You’d have to ask someone who knows more about docker than I do :stuck_out_tongue:

From the systemd ticket it sounds like they made a change to support this very thing.

And to me it sounds like not every mount would need this - at least from systemd v249.

But now when trying this with systemd.services.docker.serviceConfig.LogLevelMax = 0; it ends up under [Service]. Which is wrong. And I don’t see a way to add a [Mount] as “serviceConfig” seems to be the wrong level. It probably should really be

systemd.services.docker.config.Mount.LogLevelMax = 0;

and

systemd.services.docker.config.Service.Whatever = 0;

But it seems like I would need to use a lower level API

  systemd.units.docker = {
    overrideStrategy = "asDropin"; # is there an append?
    text = ''
      [Mount]
      LogLevelMax=0
    '';
  };

but I just cannot find any docs on this and I was looking here for the systemd package sources but they must be somewhere else.

It seems on an Ubuntu system this helped:

  1. sudo vim /etc/systemd/journald.conf
  2. Find, uncomment and change the parameters: MaxLevelStore=notice MaxLevelSyslog=notice
  3. sudo systemctl restart systemd-journald

So on NixOS this translates to

services.journald.extraConfig = ''
  MaxLevelStore=notice
  MaxLevelSyslog=notice
''

You cannot put a [Mount] section in a service unit. A unit only has three sections; [Unit] for the options that can be applied to any unit, [<TYPE>] for options that apply to that specific unit type, and [Install] which is irrelevant on NixOS and to this problem.

You don’t apply a [Mount] section to a service. Some unit types share some of their options. For instance, mount and service units share all the options described in man systemd.exec, including LogLevelMax. So to use LogLevelMax on a service, you put it in the [Service] section.

But again, you do not need to apply any settings to the docker service unit whatsoever. That will not help. The problem is that systemd is logging when mounts succeed, which means you need something to apply to all those mount units.

Note that this applies to everything on the system.

The way I read it (and which seems wrong) was that the mounts originate from the service. And that in newer systemd versions the LogLevelMax gets inherited from the service to the mounts.

That said, setting it on the service - I cannot remember seeing it making a difference yet. Just as you suggested.

Changing it in journald itself was the only thing that helped to reduce the messy logs.

Note that this applies to everything on the system.

Indeed worth noting.

Nothing in that stackoverflow thread implies any kind of inheritance from the service, and both answers are suggesting applying drop-ins to mount units and not a service. That said, the second answer suggests that a drop-in on run-docker-.mount (note the trailing -) would apply to all descendents, and in my limited testing, that appears to be the case. So maybe try this?

  systemd.units."run-docker-.mount" = {
    overrideStrategy = "asDropin";
    text = ''                                                                                                                                                                                 
      [Mount]                                                                                                                                                                                 
      LogLevelMax=0                                                                                                                                                                           
    '';
  };

Again, the trailing - in that name is important.

1 Like