The journal-upload.conf and journal-remote.conf files exist in /run/current-system/systemd/share/factory/etc/systemd but I don’t see how they can be edited. I see the /run/current-system/systemd/lib/systemd/ binaries for the tools (which can be run directly but are not included in in the PATH) and /run/current-system/systemd/example/systemd/system/ contains service files but they’re only examples so I guess I’ll try converting them to nixos systemd service configs to install them manually…
The most substantial thing I can find on integrating these tools into NixOS is a comment on a development discussion from @arianvp on this forum: The papercut thread - post your small annoyances/confusions here - #35 by arianvp which is from a year ago and mentions work on a module to implement these as options in NixOS. If this were implemented, it would be exactly what I’m looking for! I’m wondering if @arianvp or anyone else knows what is the status of this work, or if anyone knows if there is intended/existing way for these tools to be used in NixOS? If I get something working I’ll post it back here. Thanks!
I got it working! Once they were converted to NixOS configs, I didn’t need to make too many changes to the default in the two example service files for journal-remote and journal-upload (plus a small journal-remote.socket file).
I disabled https by changing the --listen-https=-3 to --listen-http=-3 on the remote side, and used a -u http:// address on the upload side. Fortunately I’m already using a wireguard tunnel between the client and the server, so I didn’t need to worry about the seemingly buggy implementation of https.
Besides not using https, the only changes were pretty obvious things like creating users for the services (and deciding to add a group on the remote side as well), and opening a port for the listener in the firewall of the server. I also added a wantedBy = [ "multi-user.target" ]; line to the remote service so it would autostart. Not sure why it wasn’t there in the example file.
Here are the two resulting NixOS config files, one for the journal-upload.nix for the client and journal-remote.nix for the server:
I found that this recent comment on an LWN article was a good overview of some of the limitations of this setup: https://lwn.net/Articles/812458/ The only issue mentioned here that I have encountered so far is that the journald.conf log size management (through services.journald.extraConfig on NixOS), such as SystemMaxUse, do not apply to the journal files in the /var/log/journal/remote directory, which has been a known bug for a couple years now: systemd-journal-remote ignores rotation rules · Issue #5242 · systemd/systemd · GitHub
Fortunately, journalctl can still operate on these remote journal files, and a command such as journalctl -m --vacuum-size= will achieve the same goal. At some point I’m probably going to write a systemd timer to periodically run this vacuum command but haven’t done it yet.
And as I mentioned in the original post, I’m still curious to know if anyone is working on integrating this functionality into NixOS options.
Wow well done! I think you are 1 step away from contributing those module to Nixpkgs.
Edit: on a 2nd thought, perhaps considering this feature is buggy with https, it’ll be a bit unsafe to contribute it to Nixpkgs… Not all of us have already configured a VPN tunnel , and without it, your setup is not secure…