Experimental does not mean unstable: DetSys's perspective on Nix flakes

Links
20

I mean… Yes? He is the reason we got the experimental flag and this whole feature plus new cli all at once?

One of the things that has been problematic with flakes is the constant deflection of problems we encounter as “not important” by the main implementer while at the same time keeping it in experimental.

If you advocate for it so much and are so convinced this is great, then go for it man. If you do not, then sit down and actually learn from the experiment. Right now it feels more like “seagull management”.

And i know this was not the goal. If there is one thing I have no doubt about, it is that you all at DetSys believe this will be good for the world.

But this weird “it is so good for everyone it should be merged” while at the same time saying “yeah we need massive multi year work before it can be non experimental” and also telling everyone they should use it anyway…

It sends pretty mixed signals isn’t it?

15 Likes
21

If he did not we would not have that debate.

22

First off, I’m preemptively jet lagged on a plane to NixCon at the moment, sorry if this isn’t coherent or some points are covered by the article. I promise I read it a few hours ago.

I think the situation is a little bit more nuanced than “stable = hasn’t broken frequently” and “should we ship or wait for perfect?”.

I also think this discussion is a bit moot given that we have an accepted RFC about a path to stabilization. Presumably flakes would be marked “stable” at the end of that process, no?

Stability

On the first point, are flakes “stable in practice” because:

  • The fundamental design is good and we’re working on improvements around the edges?
  • We’ve been deadlocked on figuring out how to move forward?
  • There are fundamental Nix features that must be implemented before we can make user-facing changes?

These are all very different situations that present the same way to users: “I can keep doing what I’m doing and nothing breaks”. I would argue that only the first case is a good candidate for stabilization.

Another question: what do we mean when we tell a user a feature is “stable”? I would argue that it means that a feature is reliable in the sense that both the code works without the need for constant maintenance and it’s valid to build a mental model around the idea. I think given all of the drawbacks mentioned so far, the solutions could require a user to substantially change their mental model of what a flake is/does after all these issues are sorted out.

Ship it vs. keep it in the lab

Obviously there’s a spectrum here. With everyone from hobbyists to corporations depending on Nix it’s not a good idea to “move fast and break things.” It’s also unproductive to wait until something is absolutely perfect before releasing it.

I would argue that fundamental flaws that need fixing weigh heavily in favor of keeping a project in the lab. Incomplete features are fine as long as they can be completed later without requiring “too much” extra work to add the missing functionality (ie you don’t have to change major parts of the system to fix the problem you created for yourself).

I also think that a large number of issues weighs heavily towards keeping the project in the lab.

13 Likes
23

Evidence from GitHub strongly suggests that the Nix community is moving in the direction of flakes and the unified CLI en masse

Please don’t suggest that the number of people using something is in any way related to its quality. We’ve had lots of propaganda surrounding flakes, much like this very article; that’s why it’s being adopted so heavily.

And I hear from more recent Nix adopters over and over again that flakes were crucial to their learning journey.

And I, as someone who spends hours every week aiding people in chat rooms and forums, have seen massive amounts of confusion caused by flakes.

I believe that flakes are stable, and I’m not alone.

They’re “stable” because no one has bothered to make the breaking changes necessary to fix the many grievances with them.

33 Likes
24

Effectively he does have ability to veto a specific change, from what I have seen.

Popularity is effectively at odds with quality, unless maybe you can sell people on criticality of integrity at Postgres level. Flakes are a regression on design quality from previous state of Nix.

And when you market Flakes «for adoption» you end up with superficial incantations not attempts to understand a thing throughout and use based on that understanding — avoiding the risk that anything will get more consistent over time.

7 Likes
25

No, Eelco doesn’t have that authority

That might be the case today, but let’s not forget that this was clearly not the case when flakes were designed, initially merged into master, introduced everywhere despite an ongoing (and then closed) RFC.

I want to address three common criticisms of flakes.

No mention of lazy trees? oO
I think there was pretty much a consensus that this was the main issue to be addressed before dropping the experimental flag. Because flakes are absolutely unusable in large repos (I tried nix build .#hello on a dirty repo - i.e. a normal situation when working on nixpkgs and I decided that I’ll never do that again. Also the reason why I’d never sneak a flake.nix into a monorepo).

This PR completely exploded, started suffering from feature creep and… stalled. In fact it was pretty usable (to me at least, though I’m despite my criticism here a flake proponent and use them pretty heavily, so I considered that a good sign) at some point in winter last year until I got some weird breakage on each update and I had enough. Even worse, lazy-trees is the reason to revert other valid bugfixes that are… still bugs, e.g. Revert "Merge pull request #6621 from Kha/nested-follows" by edolstra · Pull Request #6983 · NixOS/nix · GitHub.

I mean, complaining about another NixCon without a clear way forward while there was a plan last year is… questionable to say the least.

But it has nothing to do with flakes and thus no bearing on stabilization.

Right now we depend on a single system attribute in several outputs (e.g. apps/packages) and the output discovery expands e.g. .#foo to .#packages.<builtins.currentSystem>.foo in nix build (briefly said).

This means that we have something that juggles around with a system field without taking cross-compilation into account at all while we skip the entire problem with the nix-* commands. Yes, that may or may not be pretty either, but with a design actively neglecting this, I don’t think it’ll be easier.

Also, this is nothing new: [WIP] Add 'perSystem' and 'systems' schema attributes by gytis-ivaskevicius · Pull Request #6773 · NixOS/nix · GitHub & Provide builtins.currentSystem as a input to flakes or make system a parameter · Issue #3843 · NixOS/nix · GitHub.

It’s a good read but I disagree with the conclusions. With evaluation caching and the functional nature of Nixpkgs, source explosion is generally mitigated using the follows mechanism built into flakes, which reduces the number of unique Nixpkgs instances involved in an evaluation.

I think the main point is the usage of import inputs.nixpkgs in each flake which will cause a massive amount of evaluated nixpkgs (and follows doesn’t work recursively, the fix was as mentioned rejected because it was too hard to mege against lazy-trees).

Also, evaluation-caching doesn’t help at all when working with the code because this cache (IIRC) only caches top-level drvs and not “intermediate” drvs in the dependency-tree, so everything will be re-evaluated in a dirty tree with fresh changes.

What you can do is e.g. builtins.foldl (nixpkgs: nixpkgs.extend) (nixpkgs.legacyPackages.${system}) [ overlay1 overlay2 overlay3 ] to use a single instantiated nixpkgs if follows is applied carefully everywhere (i.e. each flake passes a follows down to its subflakes properly). However you can’t configure nixpkgs that way (i.e. allowUnfreePredicate etc.).

prior to stabilization, as a new mechanism can increment the flake.lock version field.

At least I agree with that :wink:
See the difference to the other issues? This is something we can easily change without severe breaking changes.

I gotta say I’m rather surprised by this article. While I think we’re somewhat on the same page, i.e. I also think that flakes are a massive improvement for a lot of people (myself included - that’s why I’m using them since 2020 and have also contributed numerous fixes to it), I think that the core concerns about the current state are misrepresented in this article.

When I’ve read it first it seemed to me as if the the main message was that the only arguments from the community are shallow and only there because people dislike flakes in general. While I don’t think that this was the intention, that was my first impression. And I’m concerned that such prominent statements will only worsen the divide we’re already battling in the community.

38 Likes
26

I feel a strong urge to making breaking to changes to Flakes in order to clear up that

  1. “Experimental” means experimental

  2. Popularizing experimental features from the day they are created is an Uber-style “if we break the law long enough it will become legal” tactic, and it is not destined to work.

30 Likes
27

To be clear, that is not preferred outcome. My preferred outcome is:

  1. Flakes become a separate CLI
  2. Flakes get a separate website
  3. Flakes go to a separate repo

Ideally, I would not feel threatend by shoddy Flakes ruining the core of Nix’s good name; Ideally we would have separate brands for separated projects, and separate teams for separate projects. Then I don’t have to care what Flakes do, because there is little reputational exposure either way. People/companies that want to push Flakes for growth / “Worse is Better” reasons can do so all they want, and I don’t have to pay any attention to it.

Layering and multiple brands is the only way we’re going to keep this community bursting at the seams together.

21 Likes
28

I am heavy Flakes user, but I very much like the idea of Flakes separation.

11 Likes
29

Love it hear it! It’s the natural way for us to all get along again! :slight_smile:

6 Likes
30

No, it is to get to a point where it can be reasonably believed a new major release won’t be immediately required, and that fundamental flaws that can no longer be resolved after large scale adoption aren’t present.

Case in point: Rust editions. Rust having no good way to identify integer overflow issues at compilation time is a criticism of the language, and designs have been proposed that could fix this, but will never be implemented because backwards incompatibility in integer math behavior would be catasrophic. Rust is still great, of course, but could have offered even more robustness (or not, but the opportunity window to explore this design space has passed).

That doesn’t mean you need to wait for perfection forever, but @Infinisil shared a good hitlist of issues that need to be addressed (either by deciding the design can be adapted easily later, or by resolving them).

I also think that flake’s stabilization will be an important marketing moment. It’d be great if the more obvious and major flaws weren’t present by then, even if they could technically be resolved without breaking changes.

@grahamc 's suggestion of making sure the list of blocking issues for flakes is up-to-date is excellent. Make the roadmap to flake stabilization in terms of effort clear, and I’m sure most people here will be more happy with where things are going.

Just removing the experimental flags today seems short-sighted to me, though, as much as I can emphasize with the development burnout that makes changes like that so tempting.

12 Likes
31

FWIW, Rust declarative macro system is perpetually in “new major release is immediately required” state. It’s a very bad, buggy, and inconsistent system in its core (hygiene doesn’t work for items, name resolution rules are completely alien to anything else that’s in the language, $expr captures work outside of the token stream model, the future-proofing and jointness systems turn out to not actually work in practice as well as envisioned). It gets the job half-done though, and it was instrumental in not blocking the language on all other features.

4 Likes
32

I too was very surprised that this point was omitted. How can you make flakes stable if they are unusable in the central repository of the ecosystem?

@grahamc any comment?

8 Likes
33

(Even abysmally) poor performance on certain workloads is not a blocking issue. After fixing, things will just start working for more use cases. Does not seem like a breaking change, and even if it does require some corner case behavior change, could be easily opt-in/phased-out via some extra key/version somewhere for affected users.

All that is required is a “Limitations” section in documentation, mentioning poor performance in larger repositories.

Stabiliziation has nothing to do with feature-completeness, and only to committing to certain things for the sake and benefit of downstream users.

BTW&FWIW I definitely wouldn’t mind a different CLI command for flakes. Would save me some typing. As for website and repo, also whatever.

In a way it might be even beneficial. This whole “Nix is language, but also package manager, but there’s also an OS”, is a bit confusing already and adding “Nix Flakes” to the list doesn’t help. It might be easier to say “We’re using Flakes, go install flakes from https://flakes.org then run flake develop”, instead of “We’re using Nix flakes, so install Nix, but not the OS, just the language, but then go to a config file to enable flakes, or just use the unofficial Determinate Systems installer because it enables flakes and generally works better”.

4 Likes
34

By the way, folks: for individual issues that you feel may or should block removing the experimental label, please consider opening an issue and referencing the flake milestone here.

There’s already a flakes label. The milestone was only created recently, so instead of asking people to open new issues, you should probably just add all issues that are currently marked “flakes” and then REMOVE them from the Milestone if you can confirm that they are not blocking.

Most people, who have opened flake-related issues over the years don’t actively read every message on Discourse, so you will miss a lot of issues if you rely on the original reporters of issues to manually link their issues to the Milestone.

8 Likes
35

taking each feature and putting it through the RFC process individually sounds like a great option. i’m really tired of flakes being an all-or-nothing monolith where we just talk in circles endlessly when it comes to stabilizing it. i don’t think it’ll ever be stabilized as a whole and i don’t think it would be appropriate anyway. the scope is way too large to the point where it’s hard to even discuss because everyone is talking about different things at once.

11 Likes
36

@Infinisil I just want to thank you a lot for all your great contributions to Nix and NixOS. I have seen many of your posts here on the Discourse and to me it seems like you put a lot of thought and effort into them. You are a very valuable asset to this open source community.

Again, thank you very much. :heart:

27 Likes
37

This please. If there is something that Rust Nightly shows is that being unstable and breaking change do not stop early adopters to use it if the feature is really important to them.

So let’s use the fact that flakes are marked as experimental and break the part of the design that are blockers or limiters for some of the implementation. At the very least.

4 Likes
38

I didn’t read the article, nor replies here yet, but just from the title:

If Experimental doesn’t mean unstable, why are we talking about stabilzing flakes? Just release 3.0!

For me there are roughly three grades of stability:

  1. “experimental”: Things may break any time, in the weirdest ways, large and loudly, or silent and subtle.
  2. “unstable”: Most parts are actually stabilized, but no guarantees are given to keep them. There will be some grace or deprecation period and a proper changelog
  3. “stable”: No breaking changes that not also result on a new major release

So, yes, you are kind of right when you say “experimental does not mean unstable”, it does actually mean that you have much more freedom breaking things than the “Nix Team” is willing to do…

2 Likes
39

I don’t consider this is a problem. It’s just a situation. It has benefits. For example, more stability, accuracy when sharing binary caches…

You can configure nix to duplicate files with hard links instead of copies, so multiple versions of the same dependency not always mean duplicate disk space used. Actually, since I discovered this, I stopped worrying too much about lazy trees (which will be a benefit nevertheless; but it doesn’t seem like a flakes 2.0, but a flakes 1.1).

1 Like