Announcing Determinate Nix

Announcements
12

This post was flagged by the community and is temporarily hidden.

1 Like
13

Congratulations on your new project, just in time for the NixCon! While it’s not something I plan to use personally or advertise around me, I genuinely hope it brings you success.

That said, I can’t help but feel that the direction you’re taking might be a mistake. I understand that the primary goal may most probably be financial, but by developing a closed-source solution to address gaps in the open-source Nix ecosystem, you could be undermining your own efforts in the long run. You are working with people who have been deeply involved in open source, and specifically with this project, for over 20 years. This makes the decision even more surprising… it feels like shooting yourself in the foot and ultimately counterproductive.

You’re promoting this product in a way that seems to overlook what’s already achievable in the open-source version, and it comes across as disingenuous.

Question: Is your next step to fork nixpkgs and name it something else?

16 Likes
14

Will this not further bifurcate an already confusing eco-system?

I don’t think the name will do anyone any favors; is upstream Nix now indeterminate?

What is and is not in-scope? As your target is enterprise, what are your plans for isolated networks, if any?

4 Likes
15

20 posts were split to a new topic: NixOS foundation status quo

20

How is this different from min-free and max-free? Will those get removed in the future? Are you going to break open source nix?

3 Likes
27

Does DetSys take Nix open core?

While I understand where the “open core” reading is coming from, I should also say that the Determinate documentation is saying this:

At the moment, the Nix in Determinate Nix matches the upstream version. In the future, however, Determinate Nix will include patches that have not yet been released by the upstream project.

I think you can read that in two ways. The charitable reading would be that DetSys will introduce things into Determinate Nix which will be up-streamed later. The uncharitable reading would be that DetSys will never up-stream these changes and they remain proprietary add-ons to their new product.

But unless you jump to the uncharitable reading immediately, it is not necessary to conclude that going “open core” with Nix is the aim here.

Did Eelco do a stepping back up in the shadows?

Did anyone ever ask themselves how the NixOS Foundation should, purely from a legal perspective, continue to operate as a legal entity with nobody on there? I mean it’s not just Eelco. We had a lot of stepping down lately. So it’s also all the others except Ron. And Ron is in the US. The foundation is a legal entity in the Netherlands.

Someone has to put a stamp on things to pay the bills. The stuff that Eelco appears to have done lately and that was posted here is just that.

Again, charitable reading would be: the whole thing will be decided / reorganized / updated officially once the elections are through and we have an actual governing body.

5 Likes
33

If we may take semantic versioning as an example of a feature offered by DetSys’s SaaS platform, have we seen efforts to upstream such functionality

I will not argue that the way flakes have been handled since their inception was great - many parties being involved here, though not me - nor will I argue that flakehub is the best solution under the sun.

But these are a bit different topics in my opinion. The new Determinate Nix is neither flakehub nor flakes. It’s a different product. And from what I can see, it’s based off of the Nix daemon. But it’s not announced as a fork, rather a distribution that will include some patches relevant for DetSys’s enterprise customers.

And, again, the most chartiable reading of all this is that DetSys is trying to provide an answer to a genuine problem: Make Nix more streamlined for the enterprise needs they are trying to serve as a company and be able to react fast if additional features require changes in the code base, all the while continuing to try and upstream at least the better part of these additions.

3 Likes
34

There’s also the empirical observation that neither was the detsys installer upstreamed nor flakehub opensourced so far. One could be inclined to interpret that as an indicator in which direction this journey goes.

14 Likes
35

Steps in that direction have been taken by the installer working group, including forking the repo into the NixOS org: GitHub - NixOS/experimental-nix-installer: An experimental fork of the Determinate Nix Installer to explore upstreaming.. Ultimately, it’s the decision of the community.

4 Likes
36

At least from the outside it looks like those steps were undertaken by people who are not known to me as detsys employees (except for grahams README update)? Maybe you could elaborate a bit on what forms of material support to these efforts detsys provides?

Also curious to read what “steps in that directions” are there regarding flakehub/semver, if any?

Cheers, and thanks for all your work :slight_smile:

7 Likes
37

So, min-free/max-free, which is also configurable in nix itself?
https://nix.dev/manual/nix/2.18/command-ref/conf-file#conf-min-free

3 Likes
38

Thank for you replying.

If Determinate Systems could ensure a patch would go into NixCpp, they might as well just put it there in the first place. If they can’t, then they can’t guarantee that their patches to their NixCpp-distribution will ever make it into upstream NixCpp either.

This will most likely only cause more fragmentation.

3 Likes
39

Github provides a nice UI that allows to check the work done on the NixOS community fork of the installer. According to it, the only DetSys commit to the repo was a README fix by Graham. Maybe they helped make sense of the existing code - but it doesn’t look like DetSys put much effort into upstreaming the installer.

Not to mention the fact that there’s no need to do that to begin with. DetSys installer is already on the front page of nixos.org, it is the more known installer, and it supports both CppNix installation and DetSys Nix installation. There’s not much to actually fork and change - just changing the branding and removing DetSys Nix code paths, mostly.

For disclosure, the reason I know that is in the past few weeks, I ported and authored a bunch of patches to lix-installer, the Lix flavor of the installer. I haven’t reached out to anyone from DetSys to help me with this. I asked other Lix community members one or two questions about it, but otherwise my work was fully solo (aside from porting upstream PRs, where it’s not me who made the work to begin with). The project isn’t terribly complex: more overengineered and riddled with tech debt, if anything. It’s not too hard to fork and develop mostly- or entirely self-sufficiently. To clarify my role: I don’t hold any positions of authority or any formal roles in the Lix project, I’m just a random guy who contributes to it.

But once again, there’s no reason to do that aside from branding - DetSys installer already supports installing CppNix. Lix is a separate project that installs a separate daemon, so it explicitly cares about getting the branding right, and it also needs to have self-sustainable installer because Lix has already diverged quite a bit from CppNix, and will only grow farther. Those concerns don’t seem to apply to NixOS community at large, so shrug

3 Likes
40

Yes, you could argue that way. But I’d say this: DetSys were looking for official flake stabilization for a long time. Obviously because they were pushing flakes massively to their customer base, while they also got the Nix adoption they were looking for via flake’s popularity.

If you’re in that situation as a company, the whole flakes situation looks equally bad as it does to many contributors. But from a very different angle. From the perspective of a company like DetSys, one thing you might conclude is that it’s best to get things done on your own first, and then, maybe, upstream if it’s worth the time and energy you need to actually get them accepted.

Just judging from how this thread went, people might ask themselves if engaging in extensive communication with upstream is really worth the time value of your money.

This will most likely only cause more fragmentation.

I don’t know what that means. Centralization isn’t great either. I think independent entities and teams can pursue independent projects and develop products around Nix. I’d not say that is a bad thing, rather the contrary.

1 Like
41

There’s no installer shown on the homepage, and Download | Nix & NixOS shows the official installer, not the detsys one. And new users generally use the official installer, they do not come to know about the detsys installer until someone else mentions it.

I said something like this a year ago, but I am disappointed to hear that we are still in the situation where Eelco could push to the nix project (which he owns and founded) or to the detsys project (which he owns and cofounded) and he chooses to push to the detsys project rather than enabling detsys employees to contribute to nix. This isn’t a question of if contributions can be accepted - they can, if that was the intention. Yes, Eelco stepped down from the NixOS foundation but that’s more to do with administrative/funding tasks and isn’t relevant here (which is why that’s in a separate topic).

5 Likes
42

You just argued that the generous interpretation of this is that DetSys will be able to change DetSys NixCpp quicker for their customers, and then just upstream the changes. But since there’s no guarantee going forward that their changes will be accepted (unless DetSys is controlling upstream CppNix), that means that their Nix-distribution will diverge from upstream CppNix.

Eelco announced Flakes at NixCon in 2018, I think. Don’t tell me that the group of people now employed at DetSys couldn’t possible have worked to stabilise flakes in upstream nix sometime between then and now.

3 Likes
43

Yep, that’s about the long and short of it! We make lots of patches to Nix, they’re all PR’d directly to the upstream project. Since Eelco isn’t a BDFL or fundamentally in charge of the project, a number of them haven’t landed yet for a variety of good and valid reasons. Hopefully this helps provide more background on this (careful) phrase: “Determinate Nix will include patches that have not yet been released by the upstream project.” We’ve PR’d them, they’re just not released.

At the same time, we’re afforded certain flexibility on our release cadence and how we ship to our users. For example, our ability to do phased rollouts to users over time. This means we can ship more aggressively and react more quickly. This is how we concluded that flakes are stable as they are today and guarantee their stability into the future: using the data we’ve seen from our users and general Nix adoption.

At the end of the day, we’re not looking to diverge in ways that break compatibility. I believe the project is better with us all working together. That’s why it isn’t a fork, but a downstream distribution. Hopefully, having real-world usage and stability data about the patches we’ve PR’d upstream will make it easier to land in upstream Nix.

6 Likes
44

DetSys was clear pretty early on in the process (by Spring 2023, at least) that they didn’t feel like it was their place to “push” the Nix project to adopt their installer (i.e., people on the Nix side would need to evaluate/decide/pull and build the muscles to maintain differences). The Nix team representatives present were also clear that they didn’t have the bandwidth to do the pulling.

Lest it get stuck at that impasse, the pulling has fallen to Matthew and I (and Domen did a smidge from a hackathon). This has been slow because we are few and (at least I personally) have very little bandwidth to dedicate to it.

At least one person from DetSys shows up to each of the bi-weekly installer WG calls where they communicate what they’re up to, answer codebase/process questions, provide advice if we’re PRing something upstream, etc. They’re also receptive to any feedback we have on changes that could minimize the friction we encounter trying to follow them.

There’ve also been a few multimonth periods where it’s just made more sense to sit around and wait:

  • For much of 2023 it didn’t make sense to do much until a large refactor was baked.
  • As of late this Spring we mostly just needed to figure out an initial release process and then nudge more official teams to finally get it ~hosted under nixos.org, but the macOS Sequoia update breaking existing Nix installs consumed the focus of installer WG meetings from June through September–and it again didn’t make much sense to put effort into syncing up until it was clear that our Sequoia mitigations were going well. (I woke up early to take a look at the conflicts before work this morning, but I’ve burned that time on this message.)

The Nix team was clear that it didn’t want some of the opinionated features of the installer, so Matthew has had to work on replacing some of those parts with implementations that are equivalent to the official installer. (Since we intend to follow the DetSys upstream, this also means we have meaningful diffs to maintain during sync-up.)

14 Likes
45

That’s correct. Sorry for misinformation, should’ve double checked.

Out of curiosity: what are those opinionated features? We removed some too, but it wasn’t a large churn. But CppNix approach towards what’s “opinionated” and what’s not is likely different.

1 Like
46

Isn’t DetSys in essentially the same position as Flox? Neither company is or ever has been ‘the maker of Nix’, and both companies are offering open core products targeting enterprises where Nix itself comprises much of that ‘open core’. Each of them has an employee on the Nix team.

DetSys does not to me seem positioned to unilaterally ‘take Nix open-core’ by blocking the merging of certain kinds of features into Nix. A Nix team comprised of too many members of companies with substantial overlap in their products and strategies might risk that kind of problem, though.

It would be smart and helpful for DetSys to declare that they don’t have veto power over what goes into Nix.

It’d probably also be good for the Nix team to declare that no company does.

Similarly, it’d probably be helpful for a Steering Committee to eventually do something in conjunction with the Nix team to help show that they intend to do their best to ensure the Nix team is and will be composed such that it’s unlikely that many members will share an interest in such gatekeeping. I imagine this is an issue of which any member of such a committee is likely to be well aware. Hopefully community leaders and the Nix team can work out a good plan to help grow the Nix team as a hedge against domination by companies (or groups of companies) whose material interests in Nix improvements are narrower than those of the wider community of users and developers.

Seems doable, and basically one of the aims of a lot of recent and ongoing community work. Efforts to keep Nix free are only bolstered by friendly forks like Lix and Nixpkgs-targeting reimplementations like Tvix, too. As long as there’s a Nixpkgs/NixOS-compatible Nixlang implementation willing to accept whatever features, there’s kind of a cap on how much a company can benefit from keeping those features out of CppNix, even if they could.

There’s plenty of other places for a company like DetSys to add value or compete if Nix gets better builtins for authenticating to private repos or whatever. I think these concerns are valid and important— we’ve all seen open-core projects reject PRs to sustain their commercial developers’ respective ‘moats’. But I don’t think the overall picture here looks that much like pre-fork Terraform or GitLab or whatever, and it seems like we’re finally growing a governance structure that can work to ensure things stay that way.

7 Likes