For backups and other periodic jobs that may fail I have this set up:
systemd.services."notify-failed@" = {
description = "notify that %i has failed";
scriptArgs = "%i";
path = [ pkgs.maxwell-notify ];
script = ''
unit=$1
notify "$unit: failed. last log lines:"
journalctl -u "$unit" -o cat -n 15 | notify
'';
};
which you use by adding
onFailure = [ "notify-failed@backup.service" ];
to the service to monitor. That maxwell-notify thing is a small script that sends a message to a Matrix room.
When the backup fails I receive a message like this:
