These are all Flakes deal-breakers for me:
- As you mention, the dependency explosion problem. Except that I disagree that the follows mechanism is a good solution for that. Itās a hack and can cause various breakages. We need a version resolver. And no, I will not use your proprietary third-party project to hack around that.
- Flakes doesnāt support passing arguments from the CLI. This is a clear downgrade from the stable commands. Eelco keeps saying that configurable derivations will address this, but itās just a draft PR so far.
- The system parameter having to be passed around is awful and a clear downgrade from stable Nix.
- The
flake.nixfile not actually being a proper Nix file is a fundamental design flaw, itās very inconsistent and leads to surprises, a clear downgrade from stable Nix. - Flake sources always being copied into the store for every single change is terrible and a clear downgrade from stable Nix, which supports arbitrary granularity about which files are copied to the store. The lazy trees PR, announced last Nixcon, is meant to address this, but itās still not done and suffers from various bugs.
- Evaluation purity and evaluation caching are tightly dependent on Flakes and its references. This should absolutely not be a requirement, Nix code should be able to use these features independently of Flakes. I started a draft for a proposal here (though I havenāt had the time to make it coherent).
Just overall, the core of Nix is tainted by Flake. Nix is such a good idea, and I want Nix to take over the world, but I would be deeply disappointed if this is what we end up with.
I want to be able to use flakes, but the performance of flakes in large repositories is abysmal, and itās telling that this piece fails to mention this at all. This isnāt a ācould it be better? sure.ā level of inconvenience. I invite the author to stuff a few GB into a repo that sees heavy use and convince us that adding a dozen seconds to every single command is acceptable.
Obligatory link to @samueldrās new very relevant blog post: Nix Flakes is an experiment that did too much at onceā¦
Itās not like Nix flakes, imperfect as they are, prevent anyone from just not using them.
Some of their flaws can be fixed in backward compat. way. Some will require something like version = 2; key in the flake.nix to opt-in into a future better designs.
In the meantime flakes are awesome and onboarded tons of people into Nix.
The point of a stabilization process is precisely to avoid the need for that.
Yes, flakes are great, but there are glaring problems, and the last thing you want is to be stuck with those glaring problems because people donāt want to break backwards compatibility. If flakes are to be stabilized, things like the lazy trees issue need to be resolved first.
Itāll be awesome to onboard tons of new people, and flakes do make nix easier, but Iām not looking forward to explaining to that wave of people why every single git operation takes 10 minutes in their 50GB project directories.
And no, adoption figures donāt tape over that, thatās happening under the express understanding that flakes arenāt entirely ready. If flakes are considered stable, new users who donāt expect these kind of messy problems will quickly be driven away.
Experimental should mean unstable, there are such good opportunities to make breaking changes that would make flakes better. As it is, flakes are stalled without even being releasedā¦
I also think itās a shame that the better CLI is locked behind flakes.
It would be nice to bring those improvements to everyone.
Flakes are not the only way to handle lockfiles, nor did they need to be.
As far as Iām concerned, flakes are just a terrible lockfile spec. I see your company wants to monopolise the actual benefits of lockfiles (version bounds) but this shouldāve been part of flakes itself, if flakes are what people want.
You also forgot to mention in your article that, to date:
- The only way to handle unfree packages in flakes is with an overlay
- Thereās no (pure) way to tell the nix3 CLI to go ahead and build on an unsupported system
- Flake input url syntax is broken
- There is still no concept of development-only inputs
- Canāt reject flake config all at once, I have to manually reject each line of config, one by oneā¦
- The global registry exists and is enabled by default (although easily disabled, it encourages terrible code)
PS Isnāt Eelco part of DetSys? Heās collaborating with himself?
Sure, thatās true! DetSys has invested entire months of Eelcoās time into the lazy trees work, which should make major improvement there. Another improvement that doesnāt need to wait for experimental to be lifted.
A lot of deal breakers on flakes for people are reasons they donāt like them, but have not been dealbreakers for many thousands of people who now depend on them. Itās okay to not like them or to not use them, and also for them to have their experimental label removed to reflect the very real status: being extremely stable.
We shouldnāt break them, making them unstable.
Whatās the point? To wait forever for some perfect design and flawless implementation?
Backward incompatible versions are not an end of the world, as long as a smooth gradual upgrade path is maintained.
Case in point: Rust editions.
This one is fixable without breaking backward compatibility. And majority of project are not that large.
Popularity breeds quality, not the other way around. āWorse is better.ā By onboarding users, you give organizations and people incentive to actually improve things in their own self-interest.
Waiting forever for a project to be āperfectā is counter-productive.
This onboarding has happened via flakes already, and those folks are pretty happy with their experience :).
Iāve always wondered what this sentiment means. Can you help me understand? Yes: Eelco is a cofounder of DetSys. Do you mean to say that Eelco could just up and remove the experimental label, like heās the final authority on that?
When I started my nix journey (maybe 2y ago) I got the message that flakes were the future, so I kind of stuck my head in the sand about everything else and tried to embrace a flakes-only mineset. I didnāt realize that I had made what appears to be a political choice.
So now itās weird, because insofar as Iām capable of using nix, itās through flakes (and I like them quite a lot). But I can tell from how people approach this issue that had I been around for longer, Iād be on the other side of this: If thereās ever an opportunity to gather together and vilify a company, Iām the first one to reach for the pitchfork.
For what I want to use it for, none of flakesā limitations matter (well, except maybe this one). But if itās gonna be this rift, well now I have to worry that Iām leading people into murky waters by pointing them at a flake that I wrote from them and saying āuse thisā.
Is there something I can be doing to help prevent this rift from becoming a forever thing? Because I really like it if it could not be a forever thing.
āWe can declare flakes as stable today and find ways to address source explosion under the banner of flakes as a stable feature of Nix.ā
I think this is the most straight forward way to move things forward instead of further dividing the community on this topic. I cannot remember how many times Iāve tried to explain to people(mainly new to nixos people) why flake is still āexperimentalā but they should really just give it a go. Then thereās just waves of discussion of āto flake or not to flakeā for reasons all similar to people posted here and before. Iām really tired of this.
Sure there are many half baked or unsolved features with flake, but can we just declare them stable now and move on? Is there a way community can vote on this topic?
To me it all comes down to philosophy.
You either do thing in a ābest we can with what we haveā or for a pleasure of doing things well for its own sake. Philosophy of Haskell aka āAvoid Success At All Costsā vs philosophy of Rust. Or in other words: Makers vs poets.
When Rust decided to stabilize, it took about a year, with a clear deadline, where project was scrambling getting as much as possible to future proof it and call it stable, with an escape hatch of editions system working.
Haskell has like thousands software years and people are still writing blogs about leaving it in part because of breaking changes all the time.
The need for Nix in the industry is so strong, that it caused a huge influx of more and more members of the āmakersā tribe, that just want to use Nix productively. They donāt expect perfection. They routinely deal with software waaaaay, waaay less well designed than flakes. Itās a tough bug-eats-bug world out there.
Flakes are just something that one can onboard users to and use practically. āflakes are like Cargo.toml or package.json but for everythingā. It just makes sense. And they work. The dev shells themselves with locked dependencies are a huge upgrade, because how easy they make onboarding people to non-trivial projects. I canāt imagine not using them (or something like it) anymore.
From all the pointed out flaws only system parameter strikes me as an obvious, backward incompatibility to either accept or break now. I find it mildly annoying. Everything else seems like āweāre not ready, itās not perfect yetā.
There will be a need for āflakes 2.0ā no matter what. Either make the most important future-proofing now to make āflakes 1.0ā as good as possible and accept that better designs will either be clumsily fit into being backward compatible, or have to wait for 2.0, or the flood of motivated āmakersā are just going to more or less explicitly do it for you. Because at this point thereās already more projects and more steam behind flake-users and the fact that flakes are experimental is becoming more and more irrelevant, with people advertising and using unofficial installers that enable them by default, etc.
Is this incorrect? Seems like many of the issue I raised above are unfixed because Eelco personally vetoed them.
Also I missed an opportunity to address your core point: people depend on flakes, and some new people had great experiences with flakes, so flakes should never be broken.
I will say this: I dislike the framing that detractors are all grumpy longtime members of the community, so I will mention a couple of things:
- The biggest detractors of flakes are the ones that use it the most (if we didnāt use it at all, we would never know its issues firsthand)
- Weāre also the most invested in seeing flakes improve (after getting invested in flakes, itās not like I want to actually switch again if I can help it)
- my data point - I am new to nix (3 months into the learning journey), and last month I couldnāt figure out how to install my own (unfree) software that I developed on my own machine (not part of nixpkgs), using unfree predicate, without lying to nix about the license. (I couldnāt figure out the
overlaysoutput of flakes, tbh.) I just gave up and removedmeta.licensesfrom my repo. (Sure, now I can say āI know betterā, but it was a frustration at the time. And forcing an overlay feeds back into the source explosion issue.)
Itās also telling that even a new user like me can bump into so many of the flakes issues in only a few weeksā timeā¦
The way I see it there is no āriftā in the sense of people wanting to scrap flakes, simply those who want to see flakes be better, and those who think itās good enough bc their own use cases are satisfied.
The only thing I wonder is, if flakes are marked stable, will the issues get fixed, or will flakes suck forever? (They are still better than channels even now, but flakes vs. channels is a false dichotomy.)
No, Eelco doesnāt have that authority. The Nix team and the NixOS Foundation are a collaborative team of people of shared responsibility and authority. Eelco doesnāt hold that key. Iām quite sure if he did, it would be done by now :).
By the way, folks: for individual issues that you feel may or should block removing the experimental label, please consider opening an issue and referencing the flake milestone here. Ref: Start a Flakes milestone list Ā· Issue #8870 Ā· NixOS/nix Ā· GitHub
I meanā¦ Yes? He is the reason we got the experimental flag and this whole feature plus new cli all at once?
One of the things that has been problematic with flakes is the constant deflection of problems we encounter as ānot importantā by the main implementer while at the same time keeping it in experimental.
If you advocate for it so much and are so convinced this is great, then go for it man. If you do not, then sit down and actually learn from the experiment. Right now it feels more like āseagull managementā.
And i know this was not the goal. If there is one thing I have no doubt about, it is that you all at DetSys believe this will be good for the world.
But this weird āit is so good for everyone it should be mergedā while at the same time saying āyeah we need massive multi year work before it can be non experimentalā and also telling everyone they should use it anywayā¦
It sends pretty mixed signals isnāt it?
If he did not we would not have that debate.
First off, Iām preemptively jet lagged on a plane to NixCon at the moment, sorry if this isnāt coherent or some points are covered by the article. I promise I read it a few hours ago.
I think the situation is a little bit more nuanced than āstable = hasnāt broken frequentlyā and āshould we ship or wait for perfect?ā.
I also think this discussion is a bit moot given that we have an accepted RFC about a path to stabilization. Presumably flakes would be marked āstableā at the end of that process, no?
Stability
On the first point, are flakes āstable in practiceā because:
- The fundamental design is good and weāre working on improvements around the edges?
- Weāve been deadlocked on figuring out how to move forward?
- There are fundamental Nix features that must be implemented before we can make user-facing changes?
These are all very different situations that present the same way to users: āI can keep doing what Iām doing and nothing breaksā. I would argue that only the first case is a good candidate for stabilization.
Another question: what do we mean when we tell a user a feature is āstableā? I would argue that it means that a feature is reliable in the sense that both the code works without the need for constant maintenance and itās valid to build a mental model around the idea. I think given all of the drawbacks mentioned so far, the solutions could require a user to substantially change their mental model of what a flake is/does after all these issues are sorted out.
Ship it vs. keep it in the lab
Obviously thereās a spectrum here. With everyone from hobbyists to corporations depending on Nix itās not a good idea to āmove fast and break things.ā Itās also unproductive to wait until something is absolutely perfect before releasing it.
I would argue that fundamental flaws that need fixing weigh heavily in favor of keeping a project in the lab. Incomplete features are fine as long as they can be completed later without requiring ātoo muchā extra work to add the missing functionality (ie you donāt have to change major parts of the system to fix the problem you created for yourself).
I also think that a large number of issues weighs heavily towards keeping the project in the lab.