hmm. Do we really need to change Nix to detect overlays and do a read-through on the nix store database? OverlayFS works fine with nix-store --load-db and is a pattern that is used all over the place.
What’s wrong with just doing nix-store --load-db < /nix/.resgistration-paths at startup to register all the paths that are in the base layer as we do today with our ISOs?
In my experience, you get lots of build failures if you try to actually build packages on overlayfs. I’ve heard recently overlayfs has some mode that’s supposed to make it behave more like a normal filesystem, though — I don’t know whether I was using that or not when I tried out Nix-on-overlayfs.
But that was just the beginning, we want to give you more power to configure Repls how you would like and to increase Repl portability with other platforms. We want to give you access to an even larger collection of Nix packages: ones that were published years ago, and the latest ones published today. We also want you to be able to use Nix Flakes on Replit: it shouldn’t require extra work to configure development environments on every platform. Write a Nix Flake once and have the same reproducible environment everywhere.
To achieve this, we’re going to need a way to merge Nix stores!
I am pretty sure I miss something here, sorry if it’s obvious! What is the main difference in the use cases between the overlay fs and Nix binary cache here, is this because of they don’t invoke Nix for setting up dev environments at all?
I’ve wanted to be able to segregate a Nix store into a ‘system’ part and a ‘personal’ part, with the idea that the system store contains off-the-shelf software that wouldn’t need to be particularly protected if my computer were stolen, and the personal store contains software that is a privacy risk, IP that I’m obligated to secure, etc.; but where any paths already in the system store could be used by the personal store as needed. If I could do that by making a personal local overlay store over the system store, that would be pretty great, particularly if the stores could be handled differently in the following ways:
That looks amazing! I have one question: Will your Layered Store support be tied to specific semantics (“upper Nix” having access to entire lower store DB and can instantly substitute) or will it be pluggable? I’m thinking of the following problems:
It may not be feasible to expose the entire lower store as a Nix-compatible SQLIte database. The lower store may be large, dynamic, and even have multi-tenant awareness (e.g., Attic and nixbuild’s datalake mentioned earlier).
Future isolation mechanisms (VMs) and complex setups the may necessitate other zero-copy substitution mechanisms (e.g., virtiofs, NFS, host-initiated bind mount, etc.) that are specific to each provider.
I thought about the general problem earlier, and came up with a different idea that minimizes the awareness of the “upper Nix” regarding the substitution mechanism: Substitution Agents · GitHub
All Nix knows is that the substitution agent will somehow make the store path appear. It might run as a service inside the VM and perform virtiofs mounts, or it can be in a sidecar container and can mount new paths into the CI container’s store.
The RFC has been open for a bit, and we need more shepherds! Shepherding doesn’t take very much time for most RFCs, so we encourage anyone who would like to use this feature to sign up!
First, congrats! I’ve been excited for this since it was announced, given my own weird poor hacky attempts at doing some sort of sqlite syncing with a shared /nix/store, etc. Thanks to you and replit and reviewers, etc.
What’s the best way to get my head around this, strictly from an end-user perspective? Skimming through the RFC is a bit of a doozy, and I’m hoping I can get some working knowledge without needing to consume the whole RFC/impl details.