Better way to get secrets into systemd units?

Help
1

For getting secrets, like for example ACME SSL certificates, into units and accessible to the user running the unit, I am currently using this method.

  systemd.services.mosquitto = {

    # hack to make certificate available in service
    serviceConfig = {

      ExecStartPre = [
        ''
          ${pkgs.bash}/bin/bash -c "cp $CREDENTIALS_DIRECTORY/key /tmp/key;cp $CREDENTIALS_DIRECTORY/cert /tmp/cert"
        ''
      ];
      LoadCredential =
        [
          "key:/var/lib/acme/mosquitto.grug/key.pem"
          "cert:/var/lib/acme/mosquitto.grug/cert.pem"
        ];
      PrivateTmp = true;
      ReadOnlyPaths = mkForce [ ];
    };
  };

It works, although it feels quite hacky. And when I think about it, I am not so sure if it will actually work once ACME updates the certificate. Could probably fix that too with specifying custom ExecReload in all the systemd units, to redo the copy and reload the program in the unit.

So is there a better way? Bind mount the secrets into some accessible directory in the unit? I thought the new systemd-credentials should be the prefect solution, but most programs cannot be configured to look for environmental variables to find the path the secret is on. How are other people solving this problem?

3 Likes
2

I think BindReadOnlyPaths is indeed the way to solve this one. Assuming your problem is that you can’t make the certificate permissions match the user who is supposed to access them because you’re using DynamicUser.

An alternative is to set the Group or User to a named one, which will still dynamically generate the user in question, just ensuring that their names match whatever you have set there.

3

Also SupplementaryGroups if you don’t mind your service reading all your certs or only have 1 cert.

2 Likes
4

Do you think BindReadOnlyPaths is the best way to forward agenix secrets as well?

5

Probably, unless you’re ok with dedicating a group to this. DynamicUser is cool, but when you need shared communication/files between processes, groups are kind of the way to do that.

Note that you don’t need DynamicUser to do cgroup isolation things, so do consider it.

1 Like
6

There are other options:

  1. you can run an ExecStartPre as root even with DynamicUser like this: ExecStartPre = "!/some/binary/somewhere";
  2. alternatively, create a separate foo-config.service that runs as root which is responsible for doing all the setup.
1 Like
7

Did not know about the prefixes for ExecStart! Today I learned something!

I do not understand option 2. How is a second systemd unit supposed to set up access to credentials for another unit?

8

If it’s executed as root, and you have a DependsOn and After in the second unit, the former could create/copy files and modify permissions of those files in directories the latter unit can access.

Systemd will then change the ownership of all files in the various directories (e.g. StateDir, I believe there is one for configuration too), so you could use that to move a file containing credentials into service scope without knowing the user/group ID in advance.

9

The first service could well be something like agenix, by the way - it doesn’t have to be something created specifically for the latter.

10

Huh, apparently agenix can create individual files under a different path: https://github.com/ryantm/agenix/blob/daf42cb35b2dc614d1551e37f96406e4c4a2d3e4/modules/age.nix#L151

I imagine those will be symlinks though, would that propagate permissions correctly? Are they even accessible if you use DynamicUser? I’d imagine systemd uses cgroups magic to avoid accesses like this.

Even if it does work somehow, be careful to change the permissions to something static again in ExecStartPost to prevent UID/GID reuse giving random users access to these secrets.

11

I imagine those will be symlinks though,

they can be either, using age.secrets.<name>.symlink = false;

1 Like