I’ve been experimenting recently with running NixOS in unprivileged LXC containers and sharing the store and daemon through bind mounts. It seems to be working pretty well, even if not everything is in place to support it properly. I used bind mounts for the store, database, daemon socket and gcroots. The gcroots are tricky to handle, as they need to be modified directly by the guest : I reused the
per-container directory used for NixOS containers and used file ACLs (as
shiftfs is not packaged in NixOS).
There was a discussion about this [Nix-dev] In multi-user Nix, let the daemon handle creation of GC roots, with a suggestion of an RFC. Has there been any follow-up on the subject ? I didn’t find anything while searching in Discourse.
If we ever want to fully support sharing the store and daemon between unprivileged containers, it would probably make sense to have a way to limit the resource usage (disk and CPU time would be a good start). This could be implemented in a proxy to nix-daemon, so that we can limit complexity in the daemon itself. I suppose this would also be useful for multi-user systems, but did not find any related discussion in Discourse either.
I am interested in contributing these features, if they are considered useful by the community, but it wouldn’t be tomorrow as I have a lot of things to learn on the way.