Found this article on HackerNews:
It borrows some ideas from Nix, derivations, lazy evaluation of dependencies. They also have some features we do not have.
- They use a “fake” gcc tool chain to watch the make process and create lazy thunks as they go. This allows them to have fine-grained parallelism within a package. Thus they can easily split a package into smaller components.
Thought: there are some large packages and that take a while to compile. With slightly finer resolution, we can split a package into phases or sub-phases and distribute work better. Can we also “mock” out a particular tool and create sub-derivations in a similar manner?
- They have workers that can spin up on demand. For us this may be as simple as creating a cloud resource with Nix and appropriate accesses. NixOps can do some of this, but not quite as user-friendly as spinning up Lambda’s. They also do some interesting work where the workers directly exchange dependencies, not sure how mature that capability is, but at the very least we can consider the creation of workers on-demand.