The reason why mach-nix can do what it can do, is basically because it allows IFD.
IFD makes many things much much easier, as already mentioned by some people in this thread.
IFD provides a lot of power to nix. It allows to automate a lot more stuff, like for example reading packages metadata automatically and then acting according to it.
If we could only do IFD in nixpkgs, it would become much simpler to incorporate packages from other package universes.
So, maybe we can eliminate the drawbacks of IFD in order to comfortably use it in nixpkgs.
As far as I understand, the only serious problem with IFD is evaluation cost.
Could caching be a solution for that?
Nix 2.4 already comes with nix evaluation caching.
If this functionality would be extended a bit and an online cache made available, similar to cache.nixos.org, would this fix the problem?
If you think about it from another perspective, much of the code in nixpkgs is basically just an evaluation cache created and maintained by humans. We browse other repos, download files, read some of their metadata and then write nix expressions and put them into nixpkgs (our cache for the human IFD engine) . All manually and with very high costs. Much of it could be automated for sure.
Of course this can already be automated right now with code generators. But IFD would allow us to get rid of the code generators and also the auto-generated nix code itself. Instead, we could have some nix code which is able to properly interpret fetched packages.
I think we need to setup a CI system such as Mach-nix but for Go, Rust, and NodeJS…
Nixpkgs+hydra is already a CI system. Because this CI system is missing a feauture (IFD), you’re proposing we should make a new CI. Wouldn’t it be better to improve our existing CI and add the missing feature?
Our forked IFD-nixpkgs will sooner or later run into the same problmes as the current nixpkgs would when using IFD. Better fix the IFD issues and make it usable.
If this is not feasible, then I’m happy to support in designing some framework to bring the mach-nix approach to other languages. It will probably be much simpler for other languages than python.