Would you recommend NixOS as a server container host?

Help
1

Hi

I have a simple query, would you recommend NixOS for a server that only really runs containers like Home assistant and Nextcloud? I also use cockpit for an overview of running containers and system metrics.

I run all my containers using podman, using systemd unit files will this still work as expected?

any other considerations?

thanks in advance.

2

The exact functionality you are looking for exists as a NixOS module

I use this for some things in my homelab too; here’s an example:

virtualisation.oci-containers = {
        backend = mkForce "docker";
        containers = {
                hedgedoc = {
                        image = "quay.io/hedgedoc/hedgedoc:1.9.6";
                        volumes = [ "/var/lib/hedgedoc/uploads:/hedgedoc/public/uploads" ];
                        environmentFiles = [ "/run/secrets/CMD_DB_URL.env" ];
                        environment = {
                                CMD_DOMAIN = cfg.hostName;
                                CMD_URL_ADDPORT = "false";
                                CMD_PROTOCOL_USESSL = "true";
                                CMD_PORT = "3001";
                        };
                        dependsOn = [ "hedgedoc-postgres" ];
                        extraOptions = [ "--network=host" ];
                };
                hedgedoc-postgres = {
                        # TODO: upgrade to PG 15
                        image = "postgres:13.9-alpine";
                        ports = [ "15432:5432" ];
                        volumes = [ "/var/lib/hedgedoc/postgres:/var/lib/postgresql/data" ];
                        environmentFiles = [ "/run/secrets/CMD_DB_URL.env" ];
                };
        };
};

I forgot why I changed the default runtime from podman to docker, but both work fine.

1 Like
3

Homelab - :+1:
When you require specific metal configuration - :+1:
When you have purpose-specific usecase like k8s cluster in cloud - :-1: (generally purpose specific distros are better for this, I like bottlerocket for k8s)

1 Like
4

this is seriously interesting, so I can convert my systemd unit files and add them to the configuration.nix. is that what you’ve done here? I don’t suppose if you know whether podman auto-update will still work if its done this way? this is one of the main benefits I like about podman and systemd unit files.

for example this is one of my unit files:

# container-nextcloud-caddy.service
# autogenerated by Podman 4.4.1
# Thu Mar 23 16:50:00 GMT 2023

[Unit]
Description=Podman container-nextcloud-caddy.service
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target
After=network-online.target
RequiresMountsFor=%t/containers
BindsTo=pod-nextcloud.service
After=pod-nextcloud.service

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
TimeoutStopSec=70
ExecStart=/usr/bin/podman run \
        --cidfile=%t/%n.ctr-id \
        --cgroups=no-conmon \
        --rm \
        --pod-id-file %t/pod-nextcloud.pod-id \
        --sdnotify=conmon \
        --replace \
        --detach \
        --label io.containers.autoupdate=registry \
        --volume /home/nextcloud/caddy/caddy_data:/data:Z \
        --volume /home/nextcloud/caddy/Caddyfile:/etc/caddy/Caddyfile:Z \
        --volume /home/nextcloud/html:/var/www/html:ro,z \
        --name nextcloud-caddy docker.io/caddy:latest
ExecStop=/usr/bin/podman stop \
        --ignore -t 10 \
        --cidfile=%t/%n.ctr-id
ExecStopPost=/usr/bin/podman rm \
        -f \
        --ignore -t 10 \
        --cidfile=%t/%n.ctr-id
Type=notify
NotifyAccess=all

[Install]
WantedBy=default.target

@gytis-ivaskevicius so this is entirely for my own use so , I guess its my Homelab although I don’t tinker that much as I have Nextcloud and Home assistant (and other containers) just how I want them.
currently I run this on a fedora server, but I would prefer to switch to an immutable OS ( not Fedora core) and the only other one I was considering was Opensuse MicroOS

1 Like
5

currently I run this on a fedora server, but I would prefer to switch to an immutable OS ( not Fedora core) and the only other one I was considering was Opensuse MicroOS

Just my 2 cents: I was considering an immutable Linux (Kinoite in my case) for a server but ruled it out because system updates can require reboots. It’s very nice for a desktop, not so nice for a server. A quick look at the MicroOS docs suggests that it has the same issue.

NixOS otoh seems to never require reboots. I’m pretty new to it but it’s ability to change every aspect of the system without rebooting is just mind-boggling.

1 Like
6

I have been running podman containers on top of nix-stable for several months now. Some tidbits I’ve learned:

  • I’ve had to go back to running containers as root, but using the user="1000:100" directive in the container nix description to drop back to user privileges. I don’t believe rootless podman is very well supported yet on Nix. I had a full rootless setup previously on MicroOS (for a short time), but noticed some instability and some other pain points (reboots being one).
  • If you need podman-python for any scripts, you have to build it yourself as it’s not included in the package repository. I use it for some backup scripts.
  • I’m not sure if the auto-update flag works, but you can try it. I manage my image builds and updates with ansible.
  • Here is my podman.nix
  • Updating podman versions with the 23.05 update was somewhat painful because I had to do the system updated, which failed, then do a podman system reset, then recreate all my volumes and restore from the zfs snapshot (yay!) that I made of that pool before the update.

Otherwise it’s been running great for several months now!

1 Like
7

ah that’s a shame that rootless podman is not fully supported it’s one of the reasons I switched to podman, I take it this is only applicable when running podman containers using the nixos module ? but my understanding is that its not recommended to run containers manually with systemd unit files in /home like i currently do on fedora is that correct?

do you think this was just a caveat of 23.05 and that future updates might be less eventful ? I mean I backup my /home directory to a external drive regularly as this is where I have my config and mounts for my containers but this wouldn’t be ideal to have to restore from backup every nixos release.

I’m currently installing nixos on a spare laptop I use for tinkering, hopefully I can learn how to do things the nix way and see if auto-update works at the same time.

@biscotty

this is something that draws me more towards NixOS instead of microOS