When should docker be used over NixOS services?

1

I recently setup a partial HTPC environment with radarr, bazarr, jackett and rutorrent via virtualisation.oci-containers.containers. The process was dead simple with surface-level Docker knowledge; well done for making it so streamlined.

After everything was setup I found that most of the services are available as NixOS options, which made me ask: if a NixOS service is already available, when should docker be used?

I consider services that depend on sane/common dependencies, and only export some ports for communication as trivial. Examples are nginx, murmur, mpd, etc. Trivial services I’m alright with running as NixOS services. But non-trivial services — i.e., the HTPC environment listed above that expose web interfaces, depend on Mono/PHP, etc. — I consider running in Docker instead: because of their relative complexity I want these services only to access a bare minimum of system resources. A trivial service can also be a bad actor, but then I might as well dockerize all my services, and that offers up a lot of the convenience of using NixOS (unless there is some way to nest NixOS systems declaratively).

Thoughs? Are there any services you only run in a Docker? Why?

2 Likes
2

I haven’t figured a better rule out yet, but perhaps someone else has, the last three years?

This blog post shines some light on the area and advocates for both, but does not concern NixOS services vs. Docker.

3

I use NixOS services as much as possible, because I can use socket communication and inspect services using systemctl and journalctl instead of having to go into the containers all the time. Also, I can have a much more declarative setup using all the NixOS options. AFAIK most of the containers only have a small subset of these options that are defined by environment variables.

The only thing I use containers for is gluetun which isolates other containers in a VPN network. So gluetun and services that use it are podman containers in my setup. Gluetun was easy to set up and I decided to save time here.

4

That should be possible with containers as well.
At least in a limited form.
E.g. I can inspect the logs of my Nextcloud container with journalctl -u docker-nextcloud.service.

5

My guidelines are the following but it really depends on the application:

  • Do I need a specific version? Yes → Docker, No → NixOS
  • Do I want to follow upstream? Yes → Docker, No → NixOS
  • Do I want to do upgrades unrelated from the system? Yes → Docker, No → NixOS
  • Is it packaged for NixOS? Yes → NixOS, No → Docker

For the moment that means that the following services are running on NixOS:

  • All the webservers + certs, etc.
  • All the databases
  • HAProxy
  • RSS-Bridge und FreshRSS

And the following are running in containers:

  • Gitea (could probably be a NixOS service, very easy to update and maintaine)
  • Docker Mailserver (will get migrated to NixOS services at some point, the put too much stuff into it)
  • Plex (don’t want to deal with it)
  • Nextcloud (no ambitions to change it at the moment)
  • Grav (There is no package yet but I might build one at some point)
  • Heimdall (same as Grav)
1 Like
6

If you want to migrate your mailserver see NixOS Mailserver.

1 Like
7

Thank you I’m aware of the project but I will probably spin up my own config when I migrate.

8

NixOS services, because they are systemd services, also support some isolation features (NixOS policy regarding systemd-confinement, DynamicUser and a whole host of other options), but so far I’ve not found that particularly easy to use :slight_smile: