For any other tool, this is relatively simply addressed by pinning. I can choose when to adapt and update my project across a breaking-change boundary, and this helps minimise the extent to which I care about that tool maintaining backwards compatibility.
Many of those tools (say, for example rustc) go to great lengths and efforts to maintain that backwards compatibility, because most of their users arenât using a system like nix that helps pin the build tools (as well as the libraries), for users who need to be sure that their project from last year still builds. Thatâs always helpful, but it just helps me delay that adaptation work further when there is some update. They could save at least some of that effort, if tools like nix were more widely used and became the expected norm.
Itâs terribly ironic that nix itself canât take advantage of this, because of course you need nix to evaluate the flake pins. Itâs even more ironic that the flake file format itself is not full nix, but a restricted data form that does not support full language evaluation. This would, in theory, allow breaking that coupling, by having a much smaller and simpler mininixÂč that can be responsible for resolving which nix is needed to do the evaluation, but we ironically donât have that and instead the separation is (apparently) an awkward obstacle in the code without providing this advantage. Itâs not really clear to me whether this is an unrealised goal of the design, or an awkward consequence of the implementation, or a bit of both â but itâs clear that people see it as a flaw in both directions. For yet more irony, when building NixOS there is a step essentially the same as this, to first build the nix that nixpkgs depends on, and this mechanism has itself made testing changes to nix harder, hindering improvements to the internal architecture and resulting in unintended breaking changes; some benefits and learnings from the lix overlay have recently addressed this point.
Much like a lot of this conversation, all these ironies mean that weâre stuck in a nest of dependencies and thereâs a lot of fear about any stabilisation, any commitment that adds backward-compatibility obligations. The phrase âwe have made the boogeyman ourselvesâ applies long before any mistrust and questions about intentions enter the picture, and of course only makes those worse when they do.
What could we do, and more importantly what could we avoid doing, if there was a mininix that only dealt with finding the right nix to evaluate the rest of the flake, so that nix 4 can worry a bit less about backward compatibility overheads? Will it solve enough? Is it not useful unless we can also have an ecosystem of flakes that can be used as dependencies, evaluated with mixed versions, otherwise we have a giant python2/3 type wall of compatibility, or a giant perl5/6 wall of implementation?
The above are not new questions, of course, but we need a roadmap that has fewer boogeymen and fewer walls for them to hide behind and scare us.
1: I would call this minix but that is trademarked, ironically.
It makes it easy to misinterpret a single line. A relevant benefit of verse is that one cannot comment without addressing both the minutia and the broader point.