A call for the Nix team to present a unified front on the outcome and strategy around Nix flakes

This post was flagged by the community and is temporarily hidden.

17 Likes

I’m in that dataset yet I dislike flakes. I agree with Infinisil’s objections (and more), I wish we would redesign them completely. Yet I add a flake.nix to all my new projects. Why? Because what alternative is there right now? Flakes have sucked all the oxygen out of the room for any alternatives. So I begrudgingly go along with it all. But please don’t count my acquiescence as endorsement.

@grahamc you count the number of flake users as an endorsement: I think that’s survivorship bias and circular reasoning. You’re not counting:

  • the projects which would be using Nix if there were a better alternative to flakes, but didn’t because both default.nix and flake.nix are still too poorly designed for them
  • the people who use flakes because they prefer it to default.nix but actually dislike flakes and want a better alternative

An article linked upthread covers this exact problem: Nix Flakes is an experiment that did too much at once… — Samuel Dionne-Riel

The central thesis of that post is that flakes hamper the development of better solutions. In that context, counting flake usage as endorsement is not fair. What alternatives do we have?

38 Likes

There are some clear downsides to flakes which have been demonstrated time and time again, but they remain the only option that I think is actually worth standardizing of the ones that exist today. I actually agree that there are serious design problems with flakes, but the situation before flakes was terrible in my opinion, and users that try to use Nix for the first time wind up falling into a lot of unnecessary pain due to the amount of splitting in the ecosystem. A lot of that splitting is not going to go away until/unless flakes are considered stable. Personally, I don’t think leaning on modularity will help here, complications it would add aside, there really needs to be a module system that ships with Nix that is at least similarly advanced to flakes in my opinion. It serves an important purpose in the ecosystem.

A better design would be nice, but it’s been long enough and nothing has really emerged, and flakes themselves don’t appear to be changing substantially anymore. A couple years ago flake stabilization seemed pretty much inevitable and didn’t seem like it would even be controversial, so I am a bit disappointed that things turned out this way. I hate to be dramatic, but personally I am holding off on recommending Nix to more people until there is some reasonable resolution here, because it has become tiring to try to navigate this in a helpful way for newcomers. Having no or ad-hoc/piecemeal module management isn’t really going to cut it in my opinion.

Well, that’s my 2 cents, anyways. I’m not sure if this thread will wind up being particularly productive, but on the other hand as someone who has invested decently into flakes, the idea of flakes being removed rather than stabilized is, at worst, quite annoying. I don’t think I really disagree thoroughly on many points with people who dislike flakes on the whole, but I personally doubt there will be a perfect module system. If we have to migrate off of flakes some day in the future, then so be it, but we’re already at the point where migrating off of flakes would be hard for many of us anyways.

4 Likes

I’m in this camp as well. I use flakes because there’s things I do like about the feature. But there’s a lot I would have done differently if I had been the designer. I suspect this is at least a mildly common position.

14 Likes

There are two conversations happening around flakes in general:

  • are flakes good?
  • are flakes popular?

The original post from Grahamc is about the latter. That’s fair. He then uses that as an argument to support the former. I think that’s unfair.

This specific thread isn’t (or at least wasn’t) about discussing the actual merits of flakes; it was about their popularity, and how that is an argument in favor of stabilizing them. That argument is specious, in my opinion, and I think it’s worth pointing that out.

I actually agree that flakes are better than the previous situation. Most people do. The data shows that. The point is that “flakes or channels” is a false dichotomy; the very existence of flakes, which are a Jack of all trades but master of none, prevents the development of better alternatives. If we removed flakes and tackled the individual issues independently, we could actually come up with something even better. We wouldn’t just have to go back to channels and never try anything new again.

I think that’s what the data shows: people think flakes > channels. And that’s fair. I think that too. But that’s not an argument for “we should stabilize flakes.”

8 Likes

It’s baffling to me that we are still not really any further on with this discussion.

3 Likes

I still don’t understand why we shouldn’t stabilize flakes based on all of this, though. The way I see it, the existence and ecosystem adoption of any solution is going to be enough to discourage development of alternatives schemes. Even if an alternative scheme was produced, it would need to provide compelling enough benefits versus flakes for the existing user base to want to switch over. And while I really don’t doubt the existence of flakes has indeed discouraged some people from trying to work on alternative schemes, and I don’t deny that flakes have design flaws, I also believe a huge component of this is that a lot of problems flakes have are hard enough that it’ll be hard to make solutions that are completely better overall; there will always be trade-offs.

I think the benefit of not stabilizing flakes was that it could’ve been scrapped or substantially redesigned since it was not “stable”. However, at this point, making non-trivial breaking changes is well beyond the point of no return, and I honestly think that it would’ve been by now even without pushing from Determinate Systems et al. The story has just been more compelling for a long time, and flakes found themselves in production all over the place.

I feel bad here because it seems like flakes being stabilized is viewed as a huge loss by some, but to me it feels like an acknowledgement of a reality that has existed already. And I don’t say that to suggest that I don’t believe anyone doesn’t use flakes or to suggest that there isn’t a base of people who use it begrudgingly, but to suggest that I see the stabilization as more symbolic than an actual change in status quo. In my opinion, the experimental flags for flakes no longer serve any meaningful purpose.

11 Likes

Yeah, to me it’s a bit weird that these two things can’t coexist at least for a period: stabilizing flakes and working on fixing the underlying issues in maybe some other experimental feature. All the infighting and confusion hurts nix in more than one way I think.

7 Likes

My explanation is the fear that making flakes “stable” now, will make them even more popular to a point where it’s impossible to introduce a “flake V2” system, that’s incompatible with “flake v1”.

That’s reasonable, but I think with the problems flakes currently have, the pain will be high enough in the end to make people switch.

There’s of course to possibility to improve flakes without breaking changes, but I’d honestly prefer breaking them and make big, no-compromise changes to them and make them as good as possible. The “old flakes” should buy us some time for doing so, no?

1 Like

I like how Rust’s editions feature allows backwards compatibility, even within the same project. No reason we couldn’t in the future have that with a later edition of flakes, right?

Having said that, I’m sure the steering committee will make the right decisions once it’s established. I recommend not worrying about it, if you are.

4 Likes

On that topic, I was referred to [RFC 0137] Nix language versioning by fricklerhandwerk · Pull Request #137 · NixOS/rfcs · GitHub.

I think having proper versioning as a requirement for stabilization to allow future improvements would be sensible, however I am not sure if this can still be integrated into the current design without making backwards-incompatible changes, which at least the DetSys people strongly want to avoid.

5 Likes

We should ship what we’ve had for a few years now, shitty as it may be, and then that unblocks us to either iterate on it or fix the (numerous) other issues in the ecosystem.

The meta commentary here is that the Nix community–the loud folks that bother to post anyways–seems would rather starve to death while making the perfect sandwich than settle for something to keep them going until they can get better ingredients.

Over in the BEAM ecosystem, the late Joe Armstrong had a saying something like “I’m not asking you to boil the ocean–I just want a cup of tea.” The insistence on perfect flakes is detrimental to the ecosystem as a whole. I have observed in my interactions similar behavior in other parts of the ecosystem.

I don’t know if this is because of the growing pains of moving from an academic project to a hobbyist thing to a useful industry tool, if it’s a culture thing as people who frankly can afford to wait for a perfect sandwich are slowly getting outnumbered by people actually working in kitchens (your industry folks), or whatever else, but it’s a real problem.

14 Likes

I’m just a hobbyist – no formal training in computers – that kept hearing about nix.

I bounced off several times due to channels. I still don’t understand channels.

The most recent time I tried nix, I started with flakes from the beginning. I hadn’t done this before because I didn’t get what the word flake was (not very intuitive naming). While there is a lot more going on than just input locking, the realization that this gave nix a Cargo.toml and Cargo.lock equivalent, it immediately made more sense than channels and I haven’t looked back.

Add my vote to the “stabilize flakes” count – I found it made it easier for this newcomer to get onboarded.

11 Likes

Which way are we going?

  1. improve flakes
  2. replace flakes with something better

Neither, which I believe is the crux of the issue :grimacing: it’s been a holding pattern for the last 2-3 years. (And flakes are, I believe, 4 years old.)

1 Like

OK, is anyone presenting either (1) PRs to fix flakes or (2) a working alternative to flakes covering all use-cases?

2 Likes

Yeah, has anyone in the community actually approached this from a technical point of view to fix it? I see a lot of discussion but no real action.

6 Likes
  1. Lazy trees by edolstra · Pull Request #6530 · NixOS/nix · GitHub (2.5 years old) is meant to address one of the flakes issues (not copying unused flakes to the store needlessly, which should improve performance), and there is no PR (that I know of) to address the problems I listed above. I believe Eelco tried to provide “dev-only inputs” with subflakes, but those are even more broken and I haven’t seen news on that getting improved. For my own personal use I created a patch for a setting to reject all flake config so that my machine is not getting essentially controlled by some remote machine, so that solved one of my own issues. Unfortunately such a PR would never get accepted by nix upstream, same goes for many of the other ergonomic issues, Eelco would simply block those as he has done in the past.

  2. Depends on the use case that you care about. I care about pure eval and there is currently no PR to allow for that without flakes. Same goes for a consistent schema (though I’d argue even flakes doesn’t enforce the schema unless you manually run nix flake check). If you only care about a way to get remote code and store a corresponding lockfile rather than manually specifying fetchers or relying on channels, we have npins, nvfetcher, etc. already.

2 Likes

I loved the promise of being able to declaratively define a fleet of heterogeneous systems and development environments in a reproducible fashion.

What I got instead is a community that is stuck in infantile in-fighting, finger-pointing, snarky and passive-aggressive communication, and endless threads on various platforms about the direction of the project.

Frankly, as an end-user, I am getting seriously tired and frustrated with how the Nix project is handled and how many of the community members behave.

I’ve tried to adopt Nix for a year now. It’s been a bumpy road, to say the least.

Just a few examples I can think of:

I’ve tried to contribute to NixOS Mobile, which resides under the official NixOS umbrella as an exercise to learn Nix.

I bought a device because it was explicitly listed as supported on the official wiki, just to find out that support for the specific device was utterly broken.

The documentation on the wiki is plainly wrong and misleading. Maintainers were not helpful and seemed stressed and overwhelmed by their own project. Getting useful help was next to impossible.

From my brief experience, the problems of the mobile NixOS project seem to be representative of the whole Nix ecosystem: overwhelmed and frustrated maintainers, bad, unreliable documentation, and code that desperately needs a rewrite so that it becomes more maintainable and easier to contribute to.

Concerning flakes: As someone new to NixOS, it’s exceptionally difficult to find good official information on the advantages and disadvantages of flakes.

From the NixOS wiki:

Nix flakes provide a standard way to write Nix expressions (and therefore packages) whose dependencies are version-pinned in a lock file, improving reproducibility of Nix installations. [Emphasis mine]

This is what we get. No section that explains the disadvantages.

How am I supposed to know that Nix flakes might be a bad idea in some situations?

Oh, hold on. This is not from the official wiki. It’s just a wiki that has the NixOS name, turns up first in search results and bears the Nix logo.

Ok. Maybe the official wiki has some deeper explanation that helps someone new to Nix decide:

“Nix flakes enforce a uniform structure for Nix projects, pin versions of their dependencies in a lock file, and make it more convenient to write reproducible Nix expressions.”

Sounds great! Count me in!

Why would I not want to use flakes? The official wiki doesn’t give me any reason not to. No mention of possible disadvantages or pitfalls.

This points to a deeper problem with the Nix project. Documentation, sorry people, is abysmal at best.

This creates needless frustration and friction.

More often than not, I had to dig deep into GitHub tickets and waste countless hours to find a trivial solution for an issue that is very much known to everyone involved but hasn’t been properly documented.

At the same time, I see thousands of characters written by core maintainers of various factions “airing their grievances” about some off-hand comment someone else made in a ticket 5 years ago and how it made them feel bad or unseen, and how this or that person should be banned, unbanned or perma-banned. On and on it goes.

Imagine, all this time and energy was just spent on improving the documentation and creating a shared vision and direction for the project.

While people were arguing in yet another endless thread in this forum, it took a contributor weeks to get the cursor.sh package merged because no one had the time to review their package in a timely fashion.

It’s so utterly frustrating to witness.

14 Likes