Dropping upstream Nix from Determinate Nix Installer

6 Likes

I’m sure people will take the chance to be upset about this, but imo detsys nix is already leagues above upstream nix and enables things by default for you. If you still want upstream nix, just use the vanilla installer.

4 Likes

When does a “downstream distribution” become a fork? :thinking:

23 Likes

I use Determinate Nix and it is actually better ime than upstream nix. Parallel eval and lazy trees have been useful.

5 Likes

This is a fair question. I think people think I’m just being obnoxious when I insist Determinate Nix is a downstream distribution. But to me, it is an important and serious distinction.

I’ll re-post what I shared on Reddit:

We continuously upstream patches, and work hard to participate with and improve upstream Nix too.

We spent years with the strategy of working exclusively to improve upstream, and our most impactful changes never landed. That’s why we shifted to a downstream distribution. We need to ship these improvements, and also want them to land upstream. My perception is that forks don’t typically apply that effort. That’s why I feel this is an important and meaningful semantic difference.

To me, a fork goes off and implements their own roadmap without regard for the original project. They no longer have an “upstream”, they are the upstream. They don’t make an effort to land changes in the original, or fix bugs in the original, because they’ve created a fork. They’re the complete owners of their future. They are no longer concerned about the success of the original project.

We specifically do want Nix to adopt changes we make, and we do fix bugs upstream, and we do collaborate directly with the Nix team, and we work hard to make all of this possible because we do want Nix itself to be successful.

23 Likes

Could we get more communication from DetermineSystems’ end about the status of merging these improvements into upstream Nix?

Off the top of my head lazy trees and lazy paths have both been sitting in limbo for a very long time, and at least one of those is currently usable on Determinate Nix.

It’d also be nice to know what particular features you guys plan to upstream and what you don’t plan to upstream, and for the things you do plan to upstream, communication is essential because otherwise we end up with stale PRs that sit for months with no clue as to what’s happening.

4 Likes

Pretty much everything. I can’t think of any changes we’ve made to Nix that we don’t want upstreamed. It is hard for us to give concrete status updates on when and if something will merge, because we don’t have (and don’t want!) unilateral merge privileges on Nix.

5 Likes

This is why I would call detsys a fork: they have a version of lazy trees that they like. It’s blocked because the upstream nix team doesn’t accept it (I don’t know enough about the specifics to say what I think about the technical merits of the argument). It’s never going to land as is. There doesn’t seem to be any will afaict to compromise (from either side). There will probably be lazy trees at some point but the two implementations will not look the same.

11 Likes

One reason I personally don’t use it is that the vanilla installer doesn’t uninstall properly and tells you to go clean up some files manually with incomplete instructions - see e.g. Unclear instructions for uninstalling existing Nix · Issue #12244 · NixOS/nix · GitHub.

I believe (from hearing from other users) that determinate nix’s installer actually does, though it does other opinionated things. That likely is a factor in why it probably makes less sense to support installing upstream nix.

7 Likes

The uninstaller was the original reason I tried out the detsys installer, but eventually detsys nix had features I wanted and so I made the switch.

1 Like

With respect to the Lazy trees mention from earlier, that PR has been stalled for four months after the upstream team had issues with the implementation. From what we can see as users, there has been no communication from Determinate Systems regarding the issues upstream had with the implementation, and seemingly no interest in moving forwards with it either.

Now unless there’s a secret matrix channels where DetSys and Nix devs are communicating, I think it’s somewhat fair to come to the conclusion that the PR is stalled waiting for DetSys’ response, no? This is one example of the sort of communication we need, so that it doesn’t feel like features are forever stuck in limbo.

20 Likes

Gotcha. I’m not part of the meetings, but my understanding is there is regular communication during the regular Nix team meetings. I’ll share this feedback with Eelco, though. Thank you!

8 Likes

@DetSys: Hello, could you tell me what was the reason to drop Nix from DetNix installer? I could not find a reason in the article. Was it too hard to maintain two implementations in the same installer? Did changes in DetNix require rewriting the installer in such a way that made it hard to keep Nix available as an option as well? Something else?

Thanks.

3 Likes

Have you actually looked into the implications of the implementation of lazy trees? Like the fact that the hash part of paths of inputs is non-deterministic and absolutely random across different paths evaluations? Are you fully aware of all that this entails? E.g. Lazy trees break `builtins.unsafeGetAttrPos` · Issue #171 · DeterminateSystems/nix-src · GitHub goes on being unanswered because the design is fundamentally broken. That was highlighted during review of the upstream patch (Lazy trees v2 by edolstra · Pull Request #13225 · NixOS/nix · GitHub) and that’s why it’s in limbo and never going to be merged as is.

That is not something that nix is about IMO. Is determinism (especially store path hash part calculation) — a fundamental property and strength of the nix language — worth destroying for a quick and dirty way to solve a technical issue? The issues behind lazy trees are not communicated clearly to users and all comments on the technical side of things are met with “enterprise customers don’t care about nondeterminism”. This may be true, but other nix users don’t deserve a half-baked implementation that compromises the integrity of the language as a whole.

Since this isn’t being highlighted clearly anywhere I feel that this has to be clarified, so that everyone knows what they are getting themselves into. All these issues are in plain sight and out there for anyone to see, but they might not be immediately clear to those not familiar with the codebase.

39 Likes

The first draft of this had a few reasons, which I’m happy to share. But I want to be clear that the ultimate reason is in fact in that post:

I want my team to concentrate their efforts on giving our users the best Nix experience that we can muster.

and:

We’re targeting users who want more than just a good Nix installer: we’re targeting users who want a better Nix experience altogether .

Focus on a small team like mine is very hard, and supporting additional “user paths” makes it really hard to make everything work as well as we want. Every edge case blows up the complexity into a million more edge cases. Simply not having those edge cases makes our lives better, our users’ lives better, and our software better.

There are more tedious reasons, like user confusion. I can’t tell you how many times someone used our installer and was surprised they got upstream Nix. But they’re not as significant.

At the end of the day, we want to ship great software to users that want it. To us, the Determinate Nix Installer is great software, but only the beginning, and making it a choice users have to make sends a mixed message about what we think about our own software.

3 Likes

Hey, Graham. Understood. Thank you for the additional explanation and making this clear for me. I wish you and DetSys good luck in your endeavors, then.

4 Likes

There’s a clear double standard: contributions from outside DS are downplayed—as if only DS employees build Nix—to make the fork seem warranted. Meanwhile, DS is praised for shipping features largely developed by Eelco before he joined, and that praise is also used to justify the fork.

As I’ve said years ago, this was part of the divide and conquer strategy, so that people would create Nix forks so that effect of contributions was seems magnified.

30 Likes

IMHO, it would’ve better if the --determinate flag was turned on by default and --no-determinate was made available to install Nix from nixos/nix but I understand why it can be a hassle to support installation of multiple distributions(?) of Nix at the same time.

2 Likes