Is Nix/NixOS dying?

1

Hi,
I have been using Nix/NixOS for well over a year now. It seems that everytime I return back to the discourse, I always see drama. Whether that is the original founder of Nix leaving, Moderation team and SC, Anduril, Determinate Systems, etc.

This makes me concerned about the future of Nix. I currently rely on it for my entire infrastructure. Seeing the community slowly split apart is not something I like seeing.

Distros like Debian or Arch don’t have much (if none at all for the past couple years) drama.

I have yet to see flakes stablize. It seems to be widely adopted by the community, but it is still not consider “stable”. It also seems very ignorant to me that Determinate Nix has features that upstream Nix doesn’t have and selling points such as “Drama free” or “Stable flakes”. Combined with the high “learning curve” of Nix, beginners will hesitate using Nix.

With all of this, how can the community ensure the future of Nix?

3 Likes
2

I would say that whatever else happens, nixpkgs and nixos will continue for a long time to come. Too many people are too invested in them for anything else to happen. Nix-the-program is comparatively replaceable, however, and there’s clearly some competition in that slot. Competition isn’t a bad thing, though, so long as we keep compatibility issues manageable, which everyone seems motivated to do thus far.

Given how many nix users have an attitude that they “could never go back”, I’m also confident in the idea that nix will not die unless another system with similar properties arises to replace it.

17 Likes
3

I see more posts complaining about drama than actual drama, but I do mute all the drama categories in any case. Would be nice to find a way to mute these questions too tbh.

At best people might use guix but the inconvenience of having to learn a new lang, init system, dealing with unfree etc would not really pull most away from nix.

7 Likes
4

Yours is a valid concern and you are not the only person to voice some isomorphism of it.

My suggestion is to avoid spaces with drama and to stick to the technical spaces, e.g., Github. If you want a nice, boring, reliable version of the tooling DetNix has the incentive structure to do what you probably need (Anduril obviously does not). Upstream Nix may or may not adopt the features that matter to you, but having people you can pay to honor a contract is useful.

Also, for what it’s worth, sorry for any of your cycles wasted by community drama and toxicity.

3 Likes
5

Christ, really, here? Shame on you for using this “no drama pls” post precisely to incite it.

To answer the actual post, this is exactly why there is so much “drama”; Conflicting and dishonest messaging from corporate entities doing marketing. I have half a mind to flag that as spam, it’s just shy of obvious astroturf-style advertising.

And I'm a fool for falling for it hook-line-and-sinker!

Bold claim given how much of the drama detsys are actively and seemingly deliberately (how often can you claim incompetence before it is obviously malice?) inciting through their blog posts.

Merits of their communication aside, they are objectively pushing a specific, different (and partially closed source!) design. This is completely counter your claim, to the point that I don’t think you’ve looked much into the options or the development models of the various forks, or don’t really understand what “stability” means. That, or you’re actively doing marketing - as usual, either malicious or incompetent, makes it easy to fly under the radar with newcomers who don’t know any better and have little context, and often don’t themselves have the experience to spot incompetence outright.

If you actually want “boring” and reliable, nix has a much slower development pace and doesn’t break compatibility, unlike dix, which has broken compatibility multiple times over the last year, and has actively announced that they will be introducing further breaking changes in the near future. Even lix breaks compatibility significantly less (as is their goal, they just want to modernize the development flow, not redesign the tool).

Instability isn’t inherently bad, and FOSS isn’t inherently good. What detsys are developing is fine - however, lying about it, trying to convince newcomers that it’s the opposite of what it is, is not.

Obviously, corporate support also exists, and detsys clearly position themselves as providing that for dix. I won’t dispute that there is value to that, and I would be quite happy if they stuck to it and stopped with the false claim marketing.

To third readers; there are other consultancies around, and they can provide support for the purely FOSS nix implementations just fine. dix will - in part - lock you into detsys. I’m not going to do further analysis of the options, but I take issue with detsys using this space so dishonestly for marketing.

Stick to posting about stuff you implement (you do actually make cool contributions, I have tons of respect for some of the detsys engineers!), without trying to derail natural conversation and discrediting other contributors, and especially without lying about the merits of your offering, and we’re cool.

24 Likes
6

This implies a change happening. To me it seems a standard operating procedure here for half a decade to a decade. Maybe it’s nostalgia and it’s a decade and a half.

Note that they are unlikely to be listened to about how large a flake specification issue — and there are some — is worth a breaking change in experimental flakes that changes how you need to interact with Nixpkgs.

Like with OpenStreetMap, end-user software is replaceable in a pinch, the real question is about the database upkeep.

2 Likes
7

Nix/NixOS is not dying. For example, look at the amount of contributions here:

15 Likes
8

I would say that Nix is weirdly popular, all things considered. Amount of random projects I see that have flake.nix committed is surprisingly high. Given Nix’s quite steep learning curve, it’s really cool to see that a lot of people bother to learn it and stay using it. It doesn’t feel to me like a project that’s dying.

3 Likes
9

SC is a bank. Anduril is a US army subcontractor. These types of companies are inherently enclosed and use legacy software.
I wonder why they were on the list initially?!
I guess the motivation was just to white their reputation in the media as jurassic IT park :wink:

Nix is the next step in an evolution, just like shift from ant to maven+nexus in Java world.

I worked in a bank a bit. Even Google translate is disabled in the network. Developers were installing Eclipse - as a multipart archive sent ever mail attachment :wink:

10

SC refers to the NixOD Steering Committee here – not Standard Chartered. Though I do think they might have some Nix users in their Strats team :slight_smile:

There’s a lot of Fintech companies and banks using nix though. Mercury and Monzo both are pretty big on Nix

3 Likes
11

To bring some actually productive talk into this thread; anyone doing work around a more compliance-oriented nix-based distro?

I know sbomnix exists (and that’s the main entrypoint for finally introducing nix at the startup I work at, easy to convince people nix is a good idea if you can explain that “distroless” containers aren’t littered with software you don’t use :p), but nixpkgs is absolutely littered with cruft that should never make it onto a production system (e.g. shittier), and it’s hard to keep track of the provenance or quality (including of maintenance) of individual modules/packages.

I think there absolutely is a world in which a much reduced scope of packages and modules (similar to the set of packages tested for -small), perhaps with a closer look to tying up the hardening story, is defined and maintained a bit more strictly. That’d make it much easier to talk to these kinds of companies about deploying NixOS in production, rather than just using it for dev. A subset like that would probably help define metadata for SBOMs, too, and limit that effort to a critical subset instead of the sisyphean task it’d be today.

I even wonder if some kind of function for producing scopes of package closures for systemd services would be possible, and enable fully locked-down services the way systemd upstream appears to intend Linux security working, instead of the massive hole that is /nix/store today.

I think it’d seriously help focus maintenance attention, too, instead of bogging maintainers down with the general accretion disk of packages.

… just trying to help y’all derail this thread with food for thought, at this point we have more metadrama posts from people asking what the drama is about than actual drama.

3 Likes
12

I haven’t gotten around to trying it yet, but nix-csi does something similar. It hard links the closure to a new path and only mounts that into the container. I guess we could automatically generate oneshot services that do this before the services start… Is there a way to evaluate the closure of a systemd service?

13

Isn’t that kindof what confinement does? It sometimes misses things, but then you can add those to BindReadOnlyPaths manually.

2 Likes
14

Drama is just the noise of the underlying friction. It’s a symptom and silencing it would just mean we’d be dying in silence.

Debian, Arch etc are comparatively quiet, because they are just different schools of thought around the same basic ideas of how to put code on a computer, they’re traditional Linux Distributions.

NixOS isn’t.

I know what this is like, I also fear for my infrastructure, and know I won’t use anything but Nix for a while (because nothing compares yet).

I agree with you on flakes, I can prove it as well.

Those aren’t real features. The things they have are mostly demos and the lack of “drama” is the lack of community. Many of the things that they say “differentiate them” only differentiate the uptime and financials of the systems that mistakenly rely on them.

I can’t give a concrete proposal as to what “the community” can do. But if it made a flakes enabled by default installer, that would help. If someone added flakerefs with semantic versioning, that would be awesome. If we all just agree to call them stable, then eventually, the old guard will have to cave, and in the meanwhile, I will do my best to get them on board.

Also, fighting against the Detsys misinformation around many of these issues would be helpful. They have a blog and they have a lot of brand value having Eelco and Graham on their team. We need to have clear irrefutable documents that debunk some of the claims they make, and explain the timelines and spin they put on certain events. Doesn’t have to be part of the official project. If people Google “Determinate systems” and the first things that come up is a list of rebuttals, that would remove a lot of their power.

There are infinite other things that can be done, but if you were just looking some examples, that’s a start.

Also, drama isn’t bad, but we should stop feeding the tourist trolls that come to our forum. And stop feeding drive by twitter haters that have some hot take. They’re gone in a moment anyway, and if they go back to their distro that breaks their boot partition between updates, it’s not our problem.

And seriously. The drama is the price of relevance and growth. Doesn’t mean we shouldn’t mitigate it, but it’s not the problem. The problem is all the reasons that are causing the drama.

13 Likes
15

Yes there’s a lot of work happening. Some more coordinated than others. Most of these projects are on the highly technical side instead of in the compliance check boxing side though

Bashless Activation - Tracking Issue · Issue #428908 · NixOS/nixpkgs · GitHub recently landed which makes hardened appliance images really easy to build (e.g. AWS is using this for attestable workloads now).

There’s ongoing work on finally launching the nixpkgs security tracker.

We’ve been doing lots of stuff recently to make auditd work properly on NixOS.

There’s a lot of work being done to get NixOS runnable without setuid binaries.

I’ve been working on machine identity stuff (SPIFFE/SPIRE) and measured boot

A lot of collaboration is happening with systemd folks, the uapi group (see meeting notes: UAPI group generic documents and meeting minutes) . And we’re building on top of a lot of their work (UKIs, systemd-measure, repart, the push for suidless).

In classical open source fashion this isn’t some coordinated effort where I can give you a project board and parties working on it. It’s mostly happening by itself

6 Likes
16

When Nix team says «ah yeah, flakes have some unspecified cases where the actual evaluation behaviour is non-deterministic» and you say that flakes look stable, I think you need stronger arguments than popularity for breaking the expectations of what stable means and also ignoring all the cleanups requested in Flakes RFC and promised in the new-CLI-stabilisation RFC and actually underway now.

2 Likes
17

I’ve created longer arguments internally than this vote shows. I can however share some more polished versions of why I think this is a good idea.

And immediate refinement might be to move the goalpost. We shouldn’t declare them stable, we should declare they’re NOT experimental. That means they’d ship as enabled by default.

The “out of history” argument

In general, I think it’s telling that newcomers always wonder why flakes aren’t stable, but people that have been a long time in the project insist they remain unstable. To me, it reveals that it’s not an obvious conclusion to many outsiders, and doesn’t conform to typical industry usage of the term “unstable”.

Let’s say we lived in the distant future, we just opened the GitHub code vault, and we wanted to reboot the project. Unfortunately, the knowledge of which experimental features of Nix were ever stabilized was lost to time.

We’d probably quickly realize that things such as ca-derivations were features considered unstable, makes sense. What about these flakes and nix-command features?

A very naive approach might be to look for usage. And this metric is by no means perfect at all. But we have to start somewhere. There are 147k results for path:flake.nix on GitHub. There are 823k for path:default.nix. Okay. These were clearly some features a lot of people used.

We’ll probably quickly wonder not just about the current usage patterns, but the relative growth. Determinate Systems has already made an excellent argument here, it seems that flakes are the ones that new users would strongly prefer, and the only one with noticeable growth compared to the alternatives.

new-nix-projects-on-github
new-nix-projects-on-github712×440 52.8 KB

When we look through the actual breakage users experience in their day-to-day workflow as well, it seemed to be minimal if non-existent. While the actual nix binary did change behavior between versions sometimes, because the vast majority of users got their version of nix directly from nixpkgs, none of them ever felt that breakage.

I think we’d conclude that the feature seemed to be a widely used and powerful, and that they were probably shipped as enabled by default. The usage was clearly growing rapidly, and had clearly started to take over compared to not using flakes.

The harm reduction argument is false

Another argument that is often made is that flakes just aren’t “done” enough yet. I personally think that is perfectionism, but I think proving that it is perfectionism is not necessary, it’s sufficient to prove that them not being done doesn’t matter.

The argument that they aren’t done enough frequently leans on the idea that if we were to enable flakes by default, users of Nix would start to use an “unfinished” feature that would have all sorts of bugs.

This argument had a chance when there was only the one true nix, even if it wasn’t a good argument, but the landscape has changed. Now there is a corporate-backed alternative, that has declared that flakes aren’t just non-experimental, but stable, and guaranteed not to break. They’ve done this specifically in response to the official NixOS project’s hesitance to do so.

But here is the problem: it assumes that people will not use flakes just because they’re marked “experimental” and made harder to use. But in reality it leads to a much different outcome.

By keeping flakes experimental:

  • We alienated users that want flakes, and they gravitate towards a more closed, proprietary alternative. We are helping Determinate Systems nudge users, both personal and enterprise, towards their products instead of the official nix.
  • We provide worse support to those users that want to use flakes in our community and value open source. We are, in effect, punishing them for using flakes and not jumping ship to the closed and proprietary alternative.
  • We stall the actual real world friction that it takes to make a version of flakes that could be considered “stable”. If we started treating flakes as a first-class feature, that was installed everywhere, yes, there may be more bugs. But there would also be a much stronger, more immediate incentive to fix those issues.

Rapid Fire Counterarguments

“But the CLI is still changing!”

Removing experimental is not the same as freezing. And we could change it using major versions, new feature flags or similar. It doesn’t have to be a big-bang on/off rollout. We can make what is here and used available, and then use more granular flags for future improvements.

“We are normalizing bad design”

We are endorsing user reality. The design can be iterated on.

“You just hate Determinate Systems”

I agree strongly with their line of reasoning and arguments in many cases, at least on paper. I’ve found they’re not living up to them in practice, and I see the misalignment of incentives. The results of that misalignment are clear indicators that we wouldn’t want them to capture major parts of the project, such as the installation method, the Nix binary, “flakehub”, and most crucially getting to write the narrative of our community.

I think that given the reality of usage of flakes and nix-command, “unstable” and “experimental” are not declarations of maturity, they’re simply an unfair downgrade that makes them second class citizens.

From an outcome perspective, the harm outweighs the risk. Significantly.

8 Likes
18

(I wonder if we need to split this out into a separate thread on the stabilization proposal?)

I’m only skimming, and probably don’t have a great deal of deep knowledge here, but: Is there an argument for stabilizing a large-ish subset of the CLI and the flake file format, with deliberate carve outs for the bits we know are problematic now but that could be fixed with enough work in the medium term?

I feel perfect may be the enemy of good here. I agree the current situation is silly, but even with my fairly superficial nix skills I’ve bumped into some of the design flaws of flakes on a regular basis. At some point we’ve got to go forward or back and ‘suck up’ the problems one way or another (accept or discard.) Accepting and documenting the problems as warts may be inevitable.

Also, AFAICT Lix is planning to make flakes ‘one of many’ possible implementations of the same goals. Could we quickly / roughly split e.g. the new nix3 CLI into flake -specific and non-specific aspects? I’m sure it won’t be possible to do it perfectly, but an imperfect solution might be better at this point.

1 Like
19

Yeah, we have newcomers wonder what’s wrong with the feature, experienced users having their favourite lists of what needs to be fixed in a breaking fashion, the actual Nix team saying they are broken, and Lix carefully picking the fights but saying that flakes need to be replaced with a better solution.

So the more people know about a thing, the less on average they agree it should be granted higher status.

New users also prefer nix-ld on by default, which we don’t offer, for some reason.

Actually for the same reason — Nix is built on choosing bisectability over lowering the entry barrier.

Actually no, the newcomers won’t really find any more issues than already known.

Anything good in Nix is exactly picking objective design properties over short-term user reality.

unfair

Fair, matching what needs to be done to promise anything honestly.

second-class citizens

Note that the current level of misdesign of Flakes is to make second-class citizens out of a lot of valuable qualified work in Nixpkgs. Like cross.

Basically what Nix team is doing is stabilising CLI layer by layer, fixing the flake-and-nix3 rush job issues.

7 Likes
20

That was done umphteen years ago, you can read the RFC here: rfcs/rfcs/0136-stabilize-incrementally.md at c655bdaab40f7a467f75dbb5af4325d991874e44 · NixOS/rfcs · GitHub

The RFC was accepted, as per community RFC standards.

Detsys didn’t even comment on the RFC, despite their huge involvement and obvious interest.

Then a week after the RFC was accepted detsys did a big blog post about how flakes are stable now, using precisely this same popularity argument @cafkafk is parroting. They’ve been publishing some blog post on how flakes are stable and ready every few months since.

A large part of the animosity between detsys and the rest of the community, as well as the general distaste around flakes from longer-term community members, comes from precisely this weird situation where detsys keeps relentlessly marketing things as done, and ignoring the issues that have been clearly laid out as needing improvement.

It doesn’t help that they’ve been tacking on proprietary solutions to the standard that they forced into nix and are refusing to accept needs work.

As an aside, detsys contracting for anduril, and anduril’s… umm… business domain, obviously also hits a nerve, which definitely colors the opinion of many people, and since detsys are the primary entity pushing flakes… Well, that’s the other side of the drama coin.

Pretty much all tension I’m aware of seems to stem from these two broad issues.

History lessons aside, yeah, bits of work in exactly that direction are slowly happening, as @7c6f434c says, both in nix and lix. The nix team is pretty small, though, and a number of them work for detsys, while lix is entirely volunteer driven, so don’t expect rapid progress.

10 Likes