Hi there.
I’d like to move from CoreOS to NixOS but one thing prevents me.
I want to run containers via podman, rootless and every container or pod under an own user account.
Since podman v4.4.0 there is Quadlet integrated and with that it is very easy to deploy containers and let them run as system services. I know, I can declare container configs in my nix configuration, but Quadlet is easier (in my opinion) and will function on every OS with a recent podman.
On NixOS this is only functional with system-wide container configs deployed as root under /etc/containers/systemd. A “systemctl --user daemon-reload” with a container config under ~/.config/containers/systemd do nothing.
Typically the podman-system-generator (or quadlet-generator) is started when you “systemctl --user daemon-reload”. I can search this executable with “find” and there is one (better there are more than one) in /nix/store but it is not at it’s default location /usr/lib/systemd/system-generators/podman-system-generator
Can one explain why or what I am doing wrong?
On NixOS, everything is in /nix/store, and the default locations imposed by fhs standards are avoided as much as possible. Hence there is no /usr/lib directory at all. NixOS systems instead prefer to execute executables directly from /nix/store, and applications are linked against the directories in there instead as well.
NixOS’ raison d’être is the abolishment of the fhs.
This permits all of the benefits of NixOS - the ability to cleanly install multiple different versions of the same libraries and applications, the reproducible system configuration, the roll-backs, etc.
The downside is that it’s somewhat unexpected by newcomers who don’t read into it too much. It also breaks a lot of expectations of applications, and ensures that only binaries compiled by nix can actually run on the distro without further work.
I think this is unrelated to your issue, though. Most likely NixOS’ podman configuration simply does not support what you’re looking for yet.
I don’t know how quadlets are supposed to work under the hood, but if you can figure out exactly what’s missing it would be cool to add support. It’s probably been ommitted so far because podman/docker are generally a bit less popular around here, since systemd + NixOS basically give you everything podman can.
I doubt this a little bit. That command is just there to check for new .service files. I suspect what’s missing is some additonal user service that needs to be running for this to work.
Hi tlater.
Thank you for the detailed reply.
It seems that I have not dealt with the specifics of NixOS in great detail. I focused too much on my use case. Sorry for that.
From my understanding, it is correct, what I wrote: podman-system-generator (or quadlet-generator) is started when you “systemctl [–user] daemon-reload” (you can read here: podman-systemd.unit — Podman documentation)
With a NAME.container inside /etc/containers/systemd everything is functional. So, this is the proof, that the quadlet-generator is triggered. It only do not search in $XDG_CONFIG_HOME/containers/systemd/ or ~/.config/containers/systemd/ what it should do and what it do i.e. in CoreOS or Rocky Linux.
Something from this functionality is missing in NixOS but I do not know what exactly.
Even though NixOS is a great product, I’ll stick with CoreOS for now.
Hi all, reviving this topic as it’s something I’ve been working on recently and made some progress with, but am currently stuck near the finish line.
Essentially, I’ve been able to get the user services generating and functioning as expected, however, they will not start on system startup even with users.users.<name>.linger = true;.
Any thoughts/help would be much appreciated.
What I’ve done so far:
In order to get the systemd user-generator to work, it was as simple as putting the executables that come with the podman package into the right places:
# $HOME/.config/containers/systemd/mysleep.container
[Unit]
Description=The sleep container
After=local-fs.target
[Container]
Image=registry.access.redhat.com/ubi9-minimal:latest
Exec=sleep 1000
[Install]
# Start by default on boot
WantedBy=multi-user.target default.target
Generate it and start it as expected with: systemctl --user daemon-reload and systemctl --user start mysleep.container
The remaining issue
However, if you reboot the system, this service will not start. Only once you’ve logged in as the user will it start (again, even when linger is enabled).
Any advice on how to debug this would be much appreciated. At the very least hopefully this might help someone working through the same problems or inspire a feature update to the podman package in NixOS.
Did you actually symlink the service file into /home/user/.config/systemd/user/default.target.wants? That should be what actually starts it after boot. If I remember correctly, this has to be done manually on Nix…you can’t use the systemctl --user enable mysleep.service.
A good suggestion, but as it turns out it was something else in my system config that was causing the issue. Adding the below quote to a config, and making sure the user has users.users.<name>.linger = true; set is enough to have quadlets working on boot.
@oh-bee, if you’re still looking for a solution, this might work for you.
