That looks amazing! I have one question: Will your Layered Store support be tied to specific semantics (“upper Nix” having access to entire lower store DB and can instantly substitute) or will it be pluggable? I’m thinking of the following problems:
- It may not be feasible to expose the entire lower store as a Nix-compatible SQLIte database. The lower store may be large, dynamic, and even have multi-tenant awareness (e.g., Attic and nixbuild’s datalake mentioned earlier).
- Future isolation mechanisms (VMs) and complex setups the may necessitate other zero-copy substitution mechanisms (e.g., virtiofs, NFS, host-initiated bind mount, etc.) that are specific to each provider.
I thought about the general problem earlier, and came up with a different idea that minimizes the awareness of the “upper Nix” regarding the substitution mechanism: Substitution Agents · GitHub
All Nix knows is that the substitution agent will somehow make the store path appear. It might run as a service inside the VM and perform virtiofs mounts, or it can be in a sidecar container and can mount new paths into the CI container’s store.