Is this the current state of things?

Help
21

I consider this a gamble. Surely there are people for whom this is true, and surely there’s a huge pool of new or potential users who couldn’t care less for backwards compatibility because they have no code relying on stable features. But we also have on the order of 15 000 users (according to my estimates based on various metrics) and at least 80 companies (according to a list of the NixOS Foundation’s maintains), for some of which Nix is mission-critical. And it’s questionable how important it is to merely have more users, because arguably users who don’t contribute are a “liability” for the ecosystem: they produce more bug reports or questions than they resolve, binding maintainers’ attention – a scarce resource, actually our only scarce resource – that cannot be used for other things.

I would like everyone to be realistic about what one can actually get done with Nix(OS). It’s certainly one of the most powerful software systems in the world, but many things are far from finished, polished, pleasant to use, or economical to maintain long-term. As I see it, the reason that many people, like you, are “bought-in” to flakes is that the feature was literally advertised and “sold”. This is not how technology and science should work in my opinion. I would have preferred to see all that energy invested into stabilising the overall design (or implementing alternatives), improving testing, cleaning up the code, and writing technical documentation so users and contributors can convince themselves what’s worth investing into.

This is not to say that “progress” is impossible or undesirable. We just have to be clear about what our goals and priorities are, what “progress” is even supposed to mean, and weigh that against our individual and collective capacity to make good on any such desires.

Related: Things You Should Never Do, Part I – Joel on Software

9 Likes
22

Thanks for the reply. I’ve learned a lot in this thread and since I made the post, I’ve decided to go all in on flakes, but I’m avoiding all other helping libraries like flake-parts and flake-utils, and instead just writing my own helper functions to achieve what those libraries do. This isn’t supposed to be a critique of flake-parts or flake-utils. I see now what problems they are fixing, and they do a great job. I’m just trying to take @fricklerhandwerk 's suggestion to use “only what you actually understand and can fix yourself”.

I think we’ll get there eventually. Right now Nix can be a difficult tool to learn, and I think that is ok. I found Nix because I was having very nuanced problems with my package manager (apt at the time) that not very many people would actually care about, and I started looking around to find better solutions to dependency handling. When I found Nix I pretty quickly recognized it as a solution to my problems. I never expected it to be a simple drag-and-drop GUI or something like that. Sometimes technical problems require technical solutions. That’s part of growing and moving forward, and the joy of peeling back the layers of the Nix technology can be very rewarding.

I will say that, while I do find many aspects of Nix to be difficult to understand at first, once I understand them, I’m almost always amazed at how elegant and thoughtful the solution is in hindsight.

5 Likes
23

I agree with most, and the points about companies depending on nix and focus of nix-team resources are indeed very important to make. I’d hazard a guess though that many that rely on nix(OS) for more or less mission critical systems might actually be using flakes too.

Just brainstorming here: if one would agree that the concept of flakes is actually the future (and its problems mostly implementation-related), could it be an alternative to at some point simply call the “status quo” stable and only spend efforts on necessary backports, and focus most innovative/dev efforts on something like nix 3.0 / flakes 2.0 or nix-ng / flakes-ng that doesn’t necessarily need to be backwards compatible?

Considering how conservative other stable distros are this would give quite some time to “freeze” status quo and focus efforts elsewhere. And given the nature of nixOS one could even run complete nixOS vs nixOS-NG installations as bootable generations for dev/test/avant-garde scenarios.

Maybe such an approach could be an implementation of:

I would have preferred to see all that energy invested into stabilising the overall design (or implementing alternatives), improving testing, cleaning up the code, and writing technical documentation so users and contributors can convince themselves what’s worth investing into.

24

That is essentially what Eelco was evidently planning for in the past 5 years, and I think this is the Joel rewrite scenario. I don’t see how we’d have the resources to support a legacy system and develop a sufficiently new one at the same time. Look where we are right now: There’s essentially two Nixes baked into one, none of which is clearly superior, none of them clearly moving forward, and most new users are (rightfully) confused. Each of these systems is too large to maintain with what seems to be a total of 20h/week, while 20 issues or pull requests are added to the queue each week. There’s reasons for all of that, and again the question is what we, collectively - maintainers, contributors, users - want from all this and what we’re willing to invest to get it.

I’ve been arguing for a long time, following what @Ericson2314 has been arguing for even longer, that Nix should be a lot smaller, such that people can build their own use cases on top of it instead of piling more responsibilities into a code base that can barely sustain itself. Just making that transition would be enough work on its own. And all that would be to merely make sure that we don’t break people’s code that relies on what’s there, and prevent more people from relying on code that we can’t possibly afford to keep maintaining forever. Before we get that sorted, I can’t even imagine how we can make upstream innovation happen responsibly. For now, innovation should be for the ecosystem to have fun with.

Take Tvix for comparison. @tazjin has been rewriting the Nix language evaluator bug for bug since 2020, and now @flokli is rewriting the Nix store bug for bug. Sure, a greenfield project allows for a better architecture and that enables more agility and leaves room for innovation, but even after all this time, Tvix is still not a drop-in replacement for Nix (maybe that’s not intended though).

7 Likes
25

I realise this is very dangerous (just thinking out loud here), but

support a legacy system

in this case seems to be the thing that prevents nix from “doing the right thing”, as in

stabilising the overall design (or implementing alternatives), improving testing, cleaning up the code, and writing technical documentation

… and maybe also allow for

Nix should be a lot smaller, such that people can build their own use cases on top of it instead of piling more responsibilities into a code base that can barely sustain itself. Just making that transition would be enough work on its own

In other words, for the sake of argument, if one were to really put the former (“legacy system”) on the backburner, in favour of the latter, there might be more capacity for the arguably needed “modernisation drive”?

Very dangerous indeed since this would basically constitute “abandoning” part of the ecosystem in its current form. But, as a thought experiment: say all core contributors of nix and top-level architecture would abandon “nix-legacy” and focus on “nix-ng”, that would leave us with status quo for nix and nixpkgs for, say, a year, or two? Non-core team could still be updating packages in nixpkgs, and (I think) actually many prolific nixpkgs maintainers are not necessarily deeply involved in nix and NixOS modules. So one would “get by” in the meantime with nix as-is, and nixpkgs being partly in “maintenance mode” (stable/backports) and partly community-updated (unstable).

Obviously the glaring risk is the time estimate above, so I’m not necessarily advocating for this but just wanted to make such thoughts part of the discussion and get a feeling for the overall strategy and sentiments in the core-community.

Just as backround: I’m basically already betting full-on on Nix(OS) for my personal and my startup’s infrastructure/dev needs, so on one hand I’m dependent on (some) stability, but on the other hand I’m very much interested in nix(OS) becoming maximally robust and future-proof, and I’d surely be willing to put up with some hardships in transitioning if it would serve the long-term goals.

26

@ppenguin there are, I’d say, 3 things here, not 2:

  1. Technical quality deep within the bowls of Nix
  2. Next gen User-facing “icing on the cake stuff”
  3. Legacy maintenance

My view, and @fricklerhandwerk’s, is that 3 and 1 are the priorities, and 2 can be done by other people. So it’s not like we are abandoning the brave new world, but we’re trying to lay the foundation while doing maintenance, and someone else can erect a building on top.

6 Likes
27

@Ericson2314 Thanks for your clarification. I just got the impression from @fricklerhandwerk that (mostly) regarding (1) there’s still challenges to decide the focus and the direction, as I perceive/understand it mainly between “let’s organically improve” vs. “let’s try to do it right (more radically) with what we have learnt until now”, where the second option might very well mean to (have to) give up (some?) backwards compatibility. And that there’s no resources to do both.

2 Likes
28

@ppenguin Oh there is certainly a person-power shortage — we could absolutely do more with more people. I still think it’s good to split of (2) because there are advantages to separate autonomous times (vs trying to get arbitrary many people working in lockstep).

I just mean we’re still thinking about what to do for the more radical stuff and doing it slowly, rather than not at all. If people show up and put in the time, we’ll know how to proceed more quickly because we’ve thought ahead :).

6 Likes