I have (finally…) gotten around to setting up my first NixOS based server, to soon replace my two-decade-old Debian server.
Of course, a server needs a backup. Most of the server I can quickly recreate from the NixOS configuration, but of course not all the stateful parts.
So I set up borgbackup, which works fine, and it’s backing up some selected directories specific to my server (/data/something, /var/backup/mysql). But I wonder if there are futher files or directories worth backing up that are in non-obvious places.
For example, it seems that /etc/ssh/ssh_host_ed25519_key is rather valuable, and was created when installing the server. So should I back up /etc? But that seems rather odd for a declarative system, where I’d expect to re-create /etc from my NixOS configuration.
Is the ssh key just an odd exception, or is there more valuable state in /etc? Should I just back up the full directory?
And are there other surprising locations for state?
I would use nixos configuration to relocate the ssh keys somewhere else. (services.openssh.hostKeys)
There isn’t much in /etc that’s important. Other than /etc/nixos, the ssh host keys are pretty much it.
There is at least one semi-important thing in /var, though. /var/lib/nixos contains a record of the association between user/group names and ids, which could be important to making sure the uids and gids from your backups continue to be meaningful after a restoration.
I run an ephemeral root, so to an extent, I’m essentially restoring from a backup on each boot. You can see what I go to the trouble of saving in my config here.
Security is a ‘key management problem’, and encryption keys are state.
If i can ‘talk a good config’ for a moment, use zfs , and you can create continuous snapshots and rollback state too . Maybe one day nixos can integrate that more closely with the system, so i don’t have to roll my own scripts for doing this.
nixos-rebuild switch --zfs_snap_shot_state
nixos-rebuild --rollback --zfs_snap_shot_state
these command do not exists today, it’s just a bit imagineering on my part.
That’s both pretty neat! But I think I’ll stick close to the default nixos experience for now, because I for now I want to “own” as little complexity as possible, and hope that maybe in the future NixOS provides such an experience out of the box.