Many services can be deployed with nixos systemd services by using service.xyz.enable = true;. Now I’m wondering if and how these services could be deployed in an LXC or Docker container instead?
The reason is that I’m currently running a few services (mail, some web sites, etc.) on a very small vps. But the vps has become too small so I rented a hetzner server.
Now I’m wondering if instead of running the services on bare metal or creating and mainting vms for every service, is there a reasonable way to migrate to Docker/LXC? My “dream” would be to be able to have a host system with debian (reliable & secure) and using just nix to generate/deploy Containers for all the Services I run.
I know there are the dockerTools which can be used to build docker images but as far as I understand using NixOS options don’t work with it. And I’m also not sure if using systemd Services inside Docker makes sense. I’ve read that docker containers usually have an extremely minimal init system to keep them small.
For most Services also exist official images, but I’m kinda hesistant because a) I’d like to keep nix for configuration and b) I don’t reeaally trust them that much in terms of being really up to date
As you guess, this is tricky because most nixos modules are configured with systemd. For some things you might be able to replace the executed binary with a specially crafted script that will set up the container just right so systemd will execute it correctly, but this will be painful and probably break a lot.
Docker images can generally be trusted to stay up-to-date, especially if you use official images from the upstream projects, or images maintained by the docker community. Those are quite frankly more likely to stay up-to-date than NixOS modules, just because they’re used by far more people…
If you’re concerned about keeping your dockertools references up-to-date, look at something like nvfetcher (seems it doesn’t support docker atm, guess that’s another point on my todo list), or use podman and its auto-update features.
You can still keep most configuration in nix if you use dockertools, because docker images are configured with environment variables, and sometimes text files that you mount into their file system. Both of these things can be done handily in nix.
Thanks for the explanation and the links, I’ll check them out! Keeping stuff up to date with a nix config would’ve been a follow up question so your link to nvfetcher is perfect!
Networking is much more manual, but otherwise things Just Work
@afontaine Thanks a lot for mentioning nixos-container, I somehow missed that when going through the documentation! Do you know if it there is some way to run these nixos-container commands, or at least the build command on non-nixos systems using systemd (debian) or in a docker container itself?
Otherwise I’ll have to switch my host system to NixOS which I’m a bit reluctant to do.
And do you know if the following warning from the NixOS Manual is very critical or if it should be no problem to use this method for “production” anyway? Since nixos services do a lot of magic I can imagine that some stuff will be run as root in such a container even if not intended…
Warning: Currently, NixOS containers are not perfectly isolated from the host system. This means that a user with root access to the container can do things that affect the host. So you should not give container root access to untrusted users.
Not @afontaine, but in terms of non-nixos systems - I haven’t tried it on non-nixos, but these containers use systemd-nspawn under the hood. In theory that should also work on other OSes, though you won’t be able to run these containers with docker or podman.
As for building, the source for the script lives here. I think it only uses some nix paths, and assumes it can happily write to /var/lib/container, so I think in theory it can run elsewhere, albeit without adhering to other platform’s file system hierarchy.
It might be a case of “try and see”, but the nixos refers to what’s in the container, not the host
Personally I gave up on them after fighting with networking for a few hours - it’s tricky to set up internal networks for things like database containers.
The note you mention is mostly a warning for people expecting docker containers; It’s no different from running the modules directly on the host in terms of root permissions. Docker containers aren’t perfect with this either, though it’s explicitly a bug there.
The trade-off is fully nix based containers of course, which makes them extremely flexible without losing the convenience of the existing modules.
I should also note that dockertools does allow writing arbitrary nix derivations into containers, so you can get quite far there too. It will just require manual configuration, instead of convenient modules.
