Ordering: filesystem mount before systemd.tmpfiles (before systemd-nspawn container)

Help
1

I have a config on my NixOS VPS with three relevant components that need to run in order:

  1. A file system mount
fileSystems."/persistent" = {
  device = ...;
  fsType = ...;
  options = ...;
};
  1. A systemd-tmpfiles config relating to that mount
systemd.tmpfiles.rules = [
  "d /persistent/my-data 700 myuser myuser -"
  "d /persistent/my-data-backup 700 myuser myuser -"
];
  1. A systemd-nspawn container that binds those directories
containers.my-service = {
  autoStart = true;
  ephemeral = true;

  bindMounts = {
    "/some/dir" = {
      hostPath = "/persistent/my-data";
      isReadOnly = false;
    };

    "/another/dir" = {
      hostPath = "/persistent/my-data-backup";
      isReadOnly = false;
    };
  };

  config = ...
};

How can I guarantee that these will run in order (if possible/necessary)? I know all three of these have systemd units. Is it sufficient / a good idea to override each of the units with systemd ordering directives? (See e.g. this SE issue.)

1 Like
2

You can see the dependencies of a service at runtime with systemctl list-dependencies --after container@my-service.service. In this case, it will be ordered after mounts and tmpfiles because it is a service unit and it has the default dependencies of service units described in man systemd.service, which includes After=sysinit.target, which itself is ordered after both local-fs.target and systemd-tmpfiles-setup.service.

2 Likes
3

Thanks @ElvishJerricco! That guarantees 3 runs after 1 & 2, but is 2 (tmpfiles) guaranteed to run after 1 (mounting)? I don’t see that in the systemd dependency tree using the command you gave.

EDIT: never mind, silly mistake on my part. Configuring it the way I said I did does result in the correct dependency tree.

1 Like
4

Actually, there still does seem to be an issue. I see

[username@nixos:~]$ sudo systemctl list-dependencies --after systemd-tmpfiles-setup.service
systemd-tmpfiles-setup.service
...
● └─local-fs.target
●   ├─-.mount
●   ├─persistent.mount
●   ├─run-keys.mount
●   ├─run-user-1000.mount
●   ├─run-wrappers.mount
○   ├─systemd-fsck-root.service
●   ├─systemd-growfs-root.service
●   ├─systemd-remount-fs.service
●   └─local-fs-pre.target
●     ├─systemd-remount-fs.service
●     ├─systemd-tmpfiles-setup-dev-early.service
●     └─systemd-tmpfiles-setup-dev.service

But when actually booting the system the tmpfiles commands do run before before persistent.mount (the directories are created and then overmounted). How can I fix the ordering?

5

Uh that definitely should not be happening. Can you share logs?

6

Weirdly, having rebooted the system several times, I’m unable to reproduce the issue (though I’m certain it happened—I used different directory names in the tmpfiles rules specifically to test for it).

7

Hello @Invariance-NaN, I was just debugging a very similar issue and noticed this:

Is it possible that you previously had /persistent configured with the "nofail" mount option? That would exclude persistent.mount from local-fs.target (and therefore systemd-tmpfiles-setup.service could have started before /persistent was mounted).