Preface: I like some of the changes that the current Nix unstable development provides but I fear the whole changes that “we” are trying to get into 2.4 are too big already. We’ve had several parts of the code changed in drastic ways. The bug budget for a stable release has IMHO been exhausted a while ago.
I had these comments for a while already but only ever brought them up in many private conversations but never put it out there in “public”.
Here it goes:
I for one think we should circle back a bit on what flakes are supposed to fix and identify issues with the current solutions (random default.nix files…) and what approaches we can take to make the situation better. As Domen pointed out this also means defining what an experimental feature is.
A first step to agreeing on a proper Flakes implementation is to decide what it should solve. Solving “all of the problems” at once will probably not lead to an ideal situation. What are those problems anyway? Once there is an agreement on what it is we want to solve we can start experimenting with solutions and incrementally make the world a better place.
The whole RFC did go sideways and didn’t have a clear direction. The implementation changed as the RFC was going forward. Changing code while discussion is ongoing causes lots of churn on those involved. This is equally true for those trying to follow the RFC as well as those brave souls implementing it.
Flakes, as implemented right now, is trying to do many things but it doesn’t feel well rounded for usage from beginner to advanced scenarios. What was the goal here? Is a huge monorepo (with sub(-sub*)-flakes) a valid use case? Is anyone not using any of the support libraries that are required to write half-maintainable flake.nix files for more than just x86_64-linux? What does it actually include?
Since defining and implementing a standardized file format for flakes isn’t enough there is also the whole side-quest of overhauling the CLI. I am aware that flakes - in its current form - want to be as hermetic as possible and thus sets a few evaluator options to make it so. For an experimental initial implementation we might as well implement flake build as a wrapper around nix-build flake.nix --option ... -A .... until the file format and the general direction of flakes are decided and agreed upon.
In order to properly iterate on the Flakes implementation quickly (with everyone in the community having a chance) it would have been much better to start out by implementing Flakes logic in pure Nix instead of the Nix C++ codebase.
Nothing would have been wrong with import <nix/experiements/flakesN.nix> { inputs = { ... }; outputs = { ... }; until we are happy with UI & UX of flakes. For reasons of efficiency and future UX improvements it could then be moved into the Nix C++ code - if required at all. Before the actual implementation is “stable” we could have bundled experimental flake files with Nix in point releases. Importing those would also “unsupported” and should throw a warning on every single use.
Features such as lockfiles can and could be implemented as well. We already have command line tooling to generate “lockfiles” (nix-prefetch-git .. | jq .. > lock.json). Obviously in the end the tooling should be integrated with the Nix installation.
When iterating - locally - on the flakes format all you would have to do is exchange that path that the flake definition is imported from. Currently you must be fluent in C++ and understand all the indirections the code, the fetchers, the new multi-inheritance CLI classes, … are taking. At this point it should be obvious that contributing non-trivial changes to Nix requires a lot of dedication whilst we can see in Nixpkgs that we have plenty of people that are willing to write Nix.
Perhaps the lockfiles story should have been tackled after the initial version anyway. Nix users are used to pin their dependencies explicitly. Pinning dependencies also relates to the whole story around Nix having it’s own (flake) registry. Who does it serve? Does it just safe us the github:nixos/ prefix when referring to nixpkgs? Is that really worth the feature? There was a whole sub-discussion on the RFC that was not taken seriously as it was officially not part of it but yet is an essential part of how flakes fetches inputs. What is gained by introducing yet another notation for declaring inputs? Couldn’t we fix whatever was wrong with the fetchers? What is gained by having the inputs be an attribute set instead of actual Nix code? Why do we have to take them into the inputs = { ... }: function instead of using regular Nix code that used to be good enough? I feel the overall community doesn’t know why that was done or even if it is a good way it is now. It certainly serves the new CLI.
As a general note I am impressed on how well the Rust community does this. There have been RFCs to define a general direction of a feature and then there is follow-up work to actually flesh out the implementation AFTER the direction is clear. What happened here is nothing like that. We just get an RFC while we discuss an already existing implementation just to amend the implementation.