Hi! I’ve been using Nix for a little bit and frankly I love it. I’ve been running an ephemeral flake config on my main host and all the utilities and versatility of nix has really helped me learn more about linux and systems in general.
To progress a bit more (and because there’s some things I wanted to experiment with) I wanted to start moving my proxmox host’s alpine LXC’s running docker to nixos ones, and utilising modules for my services. My initial test and dive into it went well and, well, worked™. I quickly noticed something that I wanted to kind of get ahead of and that’s the store:
It’s one of the things helps nix be such a great and useful tool, at the same time having each instance isolated results in a lot of duplicate paths and applications, compounding the space used by services in comparison to how they were. So the idea was pretty obvious, they’re all running on the same hosts and storage, share a nix store via bind mounts! So I started looking into it to see if ti was viable, caveats and so on.
Basically I’ve spent a couple days looking into this, and so far an overlay store has been the most suitable, or atleast most documented, method for this. It’s also one I was able to play with and get working in a test container.
But I’ve been trying to find some information on mounted-ssh-store or even other features/methods to have a master store/host that when a slave rebuilds, it sends the commands to the master, that builds and returns what it needs, and then allows each container to use the newly built packages from the local mounted master store. A bit overkill probably but it’s interested me, and feels a bit better than an overlay store.
Unfortunately I’ve about exhausted what I can find. I haven’t been able to find anyone using nix in a similar application that wasn’t using nixpkgs vm and container tools. Other than the reference sections in the manuals, I also haven’t found much else for these features.
So does anyone have some insight, seen a similar situation work, or can point me where I can look further?
TL;DR
I want to share a nix store across linux containers and have a master host build and manage the store. Including when each container rebuilds, the master builds what it needs in place of that container and serves the store to prevent any possible duplicate packages across containers if I were to use overlay stores.