I like this thread. Sometimes conflict is bad but sometimes it’s necessary, and smart people are saying in detail why it’s necessary in this case.
10 Likes
Okay. Let me try a different approach then. Instead of arguing about definitions, can we find a consensus about harmful effects? Because that’s what I think is actually important.
The effect of Microsoft or Google extending an open web standard in a way that closes out the rest of the internet community is a degradation of the web for people who don’t use their chosen browser.
Can you make a case for how what DetSys is doing (in general, if you like, or specific to DNix) that causes a degradation of the Nix experience for people who don’t use DNix?
Does any of this degrade the Nix experience for people who don’t use DNix? Or is it just extra goodies for people who do?
Do you think that’s the wrong question? If so, can you explain how extra goodies from a competing product merit outrage in this circumstance but not others? Is it because of the overlap between maintainers, and if so—how does that work? (In my mind, problems like ‘DNix is getting cool features that Nix doesn’t have’ are fairly easily solved if the Nix team has the freedom to take what it wants from downstream (LGPL!) or not. It’s a bigger deal if DetSys is in control of the Nix team—but as I’ve argued, if that is the case then that is the root problem, not DNix, and I want the SC to step in and fix it.)
For the record, while I still think that in isolation doing PR stunts with version numbers is cringey but no more than that, the branding of ‘Determinate Nix 3.0’ is harmful because it is likely to cause confusion and I’ve edited my previous post on this point.
Can you clarify: are you assuming that the Nix team will be forced to follow DetSys’s lead on these features? Forced how—because DetSys has a controlling majority on the team, or because of community pressure? I have been arguing that the Nix team is free to make its own decisions on this point; do you think I’m wrong about that?
(I am also uneasy about this, though I think the most positive approach is to take it as motivation to make our documentation even better.)
Me neither, and I’m glad you’re still sticking your head in for what it’s worth.
2 Likes
<mod hat on>
I was a little worried what I would find in this thread this morning, and more so when i saw how many new posts there were.
Well done, folks. The discussion here is great, constructive, and exploring difficult and even existential topics. People are talking about how they feel on sensitive issues, much more than they’re telling others how to feel.
Well done.
44 Likes
I feel disappointed that Detsys has managed to dodge responding to any of the valid concerns people have shared, leaving others to defend them in their place. These kinds of discussions have happened multiple times by now - with a member of the Nix team joining in this time.
I have my own opinions on Detsys, but I do actually want to hear their point of view. Why is this not a conflict of interest? What future do you see for upstreaming these patches when upstream already has an established roadmap that you’re stepping around (see the flake stabilization RFC)? I can’t do justice to summarizing all the questions and concerns people have had, but that’s not my job. That’s their job in a proper response.
if Detsys doesn’t address this, it comes across as though they are fully aware of how the larger community feels about this announcement, and don’t care - instead using this platform to simply attract a few more customers.
I’d like to see a response. I think Detsys has a chance here to do the right thing. I hope they take it.
28 Likes
I’m glad you agree that the important question is about control!
I would also prefer a big community consensus. I would have preferred if the CppLix and Auxolotl groups (plus any that I’ve not heard about) had stuck around for community consensus too. I understand why they didn’t. Call it ‘impatience’ if you want, but I accept that not everyone is going to have the same tolerance for waiting for the best outcome when it seems like it isn’t coming any time soon or when the football has been pulled away one too many times.
So I’m asking you to exercise empathy and imagine yourself actually in the position of the people making decisions at DetSys. Not because I think the community should prioritize their interests—we shouldn’t! But we should, as part of building that big community consensus that you and I both agree is the ideal, be capable of understanding their interests well enough to consider the likely reasons why they would take the actions they’re taking. You’re under no obligation to do this, but at the very least, even if you are unshakably confident that they are hostile, more understanding will make you a more effective strategist against them.
So let’s talk about their actions:
Maybe. Maybe the strategy is to withhold features from Nix and include them in DNix as an incentive to drive users toward their product. But if that were your goal, what would you do to achieve it? Would you leave those PRs open in the upstream project, and continue to push to them as recently as three days ago? Weird strategy if so!
But instead, what if you just wanted those features to ship? What if unaddressed questions were valid reasons for holding up the feature in a project you are not exclusively the owner of anymore, but you believe in them enough that you want to ship them anyway in a project you do control? Would that not better explain the record of actions that we have?
Consider it granted that DetSys has outsize resources at its disposal.
This, though… so what? Yeah, it’s the same language that an open core project, in which both the core and the proprietary halves are controlled by the same entity, would use. But the fact that Nix and DNix are not controlled by the same entity is an important difference here!
It seems like you are concerned that DetSys does, or might, have control over Nix, by virtue of its representatives on the Nix team! That’s a reasonable concern to have! This (and the trademark question) is what I think we should focus on, instead of litigating what third parties do outside of the Nix project! What policy should this community set for circumscribing the entrepreneurial efforts of members of the Nix team? When should the SC exercise its oversight option on technical decisions made by the Nix team? Let’s hear some proposals!
I will own the use of the word ‘silly’, and I should have phrased that more open-mindedly, but I am very skeptical of the hypothesis that DetSys are trying to hold features back from Nix when the hypothesis that getting consensus on complex features in a community project is actually hard is also available. As a member of a maintainer team on other projects, I have also left some of my own PRs open for years, despite believing in them and at times being tempted to YOLO the merge button. Resisting that temptation is a virtue, I think—and at any rate, I believe Eelco would have drawn just as much ire from the gallery if he had done that instead.
‘Fear’, I did not mean in any sort of dismissive way. I described both the community and DetSys as having fears in my post, and I believe both fears to be rational.
23 Likes
My biggest issue here is this claim: “Determinate Nix 3.0 isn’t a fork of Nix—it’s a downstream distribution”.
Determinate Nix makes different choices about what features to include, and that is what makes it different to cppnix. That is not a downstream distribution. “It’s a downstream distribution” means minimal changes to get the same result, like what nixpkgs does for most packages.
I assume this happened because Determinate Systems is frustrated by the slow pace of stabilization of various features. And that’s okay! I’m frustrated too. People can disagree!* Forking the project is the Thing To Do when you disagree strongly enough, and always having that option is part of why open-source is great.
But forking has its downsides; There’s a duplication of effort. The fork has to pick a balance between maintaining compatibility with upstream and flexibility. There’s uncertainty similar to if it was a brand new project: Will it keep being maintained? Is it stable/secure/etc?
It feels like here Determinate Systems is trying to have its cake and eat it too. It is a fork (they can make different decisions about what features to include/stabilize) but also it isn’t a fork (don’t worry, this is exactly the nix you know and love). You can only pick one!
*although it’s awkward because of the people overlap between the nix team and Determine Systems, as others have mentioned
8 Likes
I’m sitting here shocked and betrayed. I haven’t assumed good faith from DetSys for a long time, since they have shown time and time again that their behaviour is hostile to the ecosystem, and I already knew this would happen when they posted a blog post calling flakes stable and yet I refused to believe this could happen.
Calling flakes “stable” means that customers of DetSys can rely on the API not changing and if it does, either DetSys has to diverge thus breaking the “not a fork” rhetoric or use its influence through Eelco to kill any initiatives changing the flake API.
Either option is awful as one creates an unneeded divide in the community and another means that DetSys can do whatever they want and community must follow, thus creating a terrible precedent
17 Likes
Note that both of these examples are projects we built on our own, and folks uninvolved with DetSys wanted to pull them upstream – which we are glad for.
5 Likes
If you have questions or concerns of what features are included into nix, feel to reach out in our bi-weekly Nix meetings: nix/maintainers/README.md at 2c1447b7f2ed523631471b9a232b462d5e7fb075 · NixOS/nix · GitHub
These meetings are usually technical. I did so far did not get the impression that Eelco hold any pull request back because of his business interest. If you think a certain pull request was overlooked, you can bring this up in a meeting or ping me in the nix matrix channel, so that I can bring this up for the next Nix meeting.
Note that there are also efforts on bringing features from determinate systems upstream: GitHub - NixOS/experimental-nix-installer: An experimental fork of the Determinate Nix Installer to explore upstreaming.
This effort was currently blocked on the infra team (where I am also involved) to add some fastly configuration . If you want to help out with that, you can also reach me out matrix and I can see how we can speed this up.
32 Likes
Hello! I’ve been watching the discussion, I think I’ll plug in my two cents.
Names
I think this question is a good place to start from:
I think the right suggestions have already been made in this thread. I’ll reiterate them. I believe it is in the interest of all parties that:
- NixOS Foundation (we now have a functioning SC and a new complete board) register a trademark, and make this a priority effort. I trust @grahamc not to interfere with this process, not to attempt registering a conflicting trademark ahead of Foundation, and not to attempt litigation.
- Determinate Systems rename Determinate Nix as to acknowledge the fact that it is an independent project, developed by an independent team on their own schedule. We can refer to many successful examples of projects that effectively reimplement what Nix does, make different choices about which features to include, and do not create conflict: Lix, TVix.
Flakes
A less obvious issue is Flakes. My opinion (someone unfamiliar with Nix codebase and development processes) is that it would be a mistake to stabilize them. I believe the best strategy wrto Flakes, despite the sunken cost, would be to call it a “make one to throw away” project. My understanding is Flakes have been a very important research project with many byproducts, closely intertwined with the work on hermetic evaluation, on CLI UX, and probably many other subjects. However, I believe Flakes as a feature are based on a misidentified problem, fail to address any fundamental issues, and fail to add value over e.g. npins. In particular, I had previously referred to @roberth’s description of Flakes as a “weak dependency injection system”, and I believe that DI was that fundamental problem that actually needed and needs solving, and which Flakes consider specifically a non-goal. I think should start phasing Flakes out. We should actively and broadly advertise Flakes as deprecated so as to hopefully alleviate some of the immense damage they have been causing to SEO and to the Documentation effort. Honestly, I think Determinate Systems might benefit from doing the same and starting over with a new name.
Some alternative or compromise strategies to ponder:
- As suggested many times before, instead of just removing them, make
flakesa separate CLI tool, developed independently in its own repository. If one were set on keeping flakes alive, I’d advise they make it a backend-agnostic tool, capable of using all major Nix-like solutions (Nix, Lix, TVix, Determinate Nix). - Regardless, the protectionist approach would be to include “Flakes” in the trademark policy, and to introduce the “Flakes” project and “Flakes” team concerned with developing and maintaining the Flakes specification.
- A more radical approach that would leave many people wary of EEE strategies is to let DetSys have Flakes. Remove Flakes from Nix. Collect signatures to have the Foundation license (IANAL) Flakes to DetSys or assign the IP to DetSys in a way that would let them register a Flakes trademark, claim ownership of the project, and start maintaining The Specification. It was suggested DetSys may need a new name for “Determinate Nix”, I propose “flakes”.
22 Likes
No. Thank you, very much. Absolutely no interest to go back to stateful channels or depend on non first-class third party solutions (npins, niv) for this.
32 Likes
I have to second @polygon on this. I am not informed about the internals of why Flakes are considered a suboptimal solution, and I also think that publishing DetNix 3.0 just like so is weird, to say the least. However, Flakes work perfectly fine for me, I am using them for all of my projects and they solve what I want to have solved: Easy specification and locking of dependencies (inputs), and also the specification of what the project at hand provides (outputs) directly using standard CppNix without dependencies.
15 Likes
fail to add value over e.g.
npins
While I agree with the statement overall, and think flakes do not belong to the nix, I can observe significant performance improvement in evaluation times of my system configurations when using flakes compared to nix-build --pure-eval. I suspect this is due to pure-eval being basically abandoned and not getting optimizations that are applied in flakes due to them being declared the only true way to do pure evaluation
This is the only reason for me to prefer flakes over npins, though, and I would much rather use that if the performance issues were resolved. I very much second the idea of phasing out flakes over time and would love for nix to focus on stability and performance issues
3 Likes
I think you should not jump to early conclusions. When folks says “phasing out Flakes”, most of the time, we are talking about removing the layering violation that is libflakes and putting Flakes on the same level as any of the alternative: npins, niv, etc. Opening up the pure evaluation knobs and more to any tool and not just that weirdly shape thing called Flakes that is mistaken for 60 features in a trenchcoat.
It doesn’t mean crippling Flakes neither and reducing its functionality, it’s cleaning up and exposing the features required for Flakes but also third party alternatives like npins and niv to thrive.
It doesn’t mean breaking Flakes for end users in the process.
Not only that, but it does mean that the end result may be a space where not only Flakes can improve but other alternatives can raise to the challenge, and we can obtain something better than Flakes, without having to deal with one individual (or entity) dictating what is their vision of Nix without any social collaboration process.
Everyone who has been repeating the narrative that “phasing out Flakes is going to cripple poor users, and it’s insane to even suggest this thing” is just falling prey to a canonical FUD strategy.
I will not comment on the larger topic at hand, but I always find it curious when I read things like “No one came up with a plan for stabilizing Flakes” : no. Many people came up with plans and ideas. At some point, you just have to stop repeating “just remove the experimentalFeatures.require(Xp::Flakes) and be done with it”, you have to work hard to get there. It’s OK that some actors can fool users into believing into how easy it is to stabilize Flakes without putting any effort and blaming other communities for it.
At some point, there’s nothing else than a dead end that awaits those who are unwilling to put efforts to build something bigger than themselves. And don’t get me wrong, even if I’m harsh, I’m grateful for the invention of Nix by their creators and all the nourishment provided by the early nixpkgs community and more.
60 Likes
Well, that is reassuring to hear, thank you. This is the part that was missing from most of the posts in these discussions. It was all just “let’s throw Flakes out, they were a mistake” without any apparent acknowledgement or even care for the effort this would cause for the users. It’s especially a bad smell that this often comes up in these DetSys threads and has a side-taste of retaliation (simplified “DetSys likes Flakes, let’s throw them out!”).
Alienating users who are not deep into Nix-dev or the conflicts going on here and creating a fear of having the rug pulled from under your feet and having to re-do countless hours of work based on these comments sure makes DetSys Nix a with their stability promises look quite a bit more sane. So it’s good that this got cleared up finally.
12 Likes
Related to what @roberth said:
Determinate Systems made very contradictory statements about what “stable” means to them. Maybe stable means to them means “we are going to change semantics as we see necessary” and provide a service to customers to deal with the changes. This would be similar to how i see what the nix team defines as unstable.
As dix and nix are two separate projects, what is deemed a necessary change will most likely be different. So both project can end up making changes that break the other one. This cannot be prevented formally but only via cooperation. If such a cooperation is interesting for either determinate system and the nix community future will show. Because I believe that the strength of open source is software quality i expect that nix-flakes will develop into a reasonable part of nix. The question if nix changes are compatible to the ecosystem of tools provided by determinate system depends on them adopting developments.
For the case that nix gets lost in “improvements” of questionable quality, there is also lix.
1 Like
Every single person using flakes has in their configuration experimental features enabled. Why should nix development be wedged forever because of that? Long before I ever started frequenting the forums it was made very clear to me by the tools themselves what was stable and what wasn’t. And because I have always been someone uninterested in unstable interfaces I never enabled flakes or added them to my workflow. It doesn’t take deep nix-community involvement at all to figure that out.
2 Likes
I do not think we are jumping to early conclusions, we were just stating that we are using Flakes and enjoy using them. Thanks for your clarification about supporting Flake-like setups even when the classical “Flakes” may be phased out!
3 Likes
Pure eval doesn’t exist outside of flakes (unfortunately), that flag is more or less a broken promise.
Personally the only reason I bother with flakes over npins/nvfetcher is the pure eval.
@elikoga the fundamental problem is non-lazy behavior in the lockfiles when the lang is otherwise (mostly) lazy, and pure eval leading to sometimes redundant copying, which in turn means bad perf in too many cases. (That’s excluding the frequent bugs due to trying to do too much.)
@dschrempf No, some use them because they are happy with them, but most use them because of detsys marketing and cargo culting, and even then it’s not literally everybody. A large percentage of the community do not use flakes per prior surveys.
I (and others) more or less had the same complaints about flakes 2 years ago, and it seems we are no closer to actually solving them.
7 Likes
I can’t believe getting lock files built into nix would take this much
11 Likes