A call for the Nix team to present a unified front on the outcome and strategy around Nix flakes

Development Nix
112

As someone who mostly likes flakes, uses flakes, recommends their use to others, and thinks they should be stable, I just hit something the other day where they weren’t: Nix 2.15 can’t read flake.lock files created by some newer versions of Nix (2.23 at most, maybe earlier versions as well). (I have not yet filed a bug report.)

I understand committing to flakes and pledging to support them, but I don’t think that obviates the need for the existing/ongoing process of stabilizing the interfaces.

1 Like
113

There is not shortcut to success here, no single quick fix.

Some relevant references.

Progress is slower than we would all like, but progress is being made. We’re somewhere around step 2 of RFC 136. The store library has been split out. Store-only CLI is possible, and commands have been added refined. Build system is refactored. The best thing at this point is help with the steps as laid out.

Jumping directly to flakes - the exciting stuff - is tempting, but help is needed with the un-exciting stuff as well. The key is incremental progress in small steps.

Edit: Additional item, a small step to fix some of the flake annoyances is a half-step toward lazy trees; lazy paths (Lazy paths by roberth · Pull Request #11367 · NixOS/nix · GitHub). I have wanted to help this get merged, but I apologize, I have not been able to do so in a timely manner.

32 Likes
114

As long as a solution is being actively worked on then I think a lot of the problems here are less warranted. Ideally, Flakes would’ve already been stabilized imo. But it seems like it’ll still be some time.

6 Likes
115

I didn’t see a single design issue in Flake under this thread. The infamous copy everything problem is actual an implementation trade off in nix cli. If you don’t need pure evaluation, you can use flake-compat and it would not copy everything.

116

In my opinion it would be a good idea to not dwell too much on design issues with flakes, since it’s neither here nor there without anything concrete to actually do about them, but I think flakes do have some pain points. It’s a hard one to solve but I’d like real version constraints and resolution, rather than just having to bruteforce override transitive dependencies to deduplicate them manually.

And of course, I’ll patiently wait the proper “stabilization” of flakes. I still think the flags are a bit pointless, since many of us have long given up on waiting and just treat it as pretty-much-stable anyways. After all, a stability promise is nice, but it’s only a promise, it’s not worth that much. And, when so much of the ecosystem is on flakes as it is, I don’t really feel afraid to rely on it… things could break, but it’s not like they’ve broken a ton in the past couple years. At this point I’m just hoping for the best.

2 Likes
117

How about those:

The last one probably can be workarounded with modules (but you force downstream users to use them), but the first two don’t really have a good solution IIRC. I’ve personally was more than once annoyed by flakes not being modular.

6 Likes
118

I’d like to invite you to try https://devenv.sh, it has a much better DX than flakes.

There’s modular support for outputs: Outputs - devenv

5 Likes
119

I’d like to invite you to try https://devenv.sh, it has a much better DX than flakes.

I need to try it for sure! So far I was under the impression that it is primarily meant for developer shell environments. It’s called devenv after all. Which would exclude things like home configurations or NixOS configurations, or any other output one might wish to have really.

But maybe learning from this improved UX could serve as a guide for how to fix things upstream?

3 Likes
120

Upstream of devenv is tvix: devenv is switching its Nix implementation to Tvix - devenv

2 Likes
121

Alright, that makes sense. That is the kind of step you finally decide to take when thinking back to your 1200km moped trip across northern Thailand instead of hanging around here? :sweat_smile:

But maybe the rest could take inspiration from that nonetheless and explore new avenues. It just seems to me after all this discussion here, which I read with no result or decision in sight, that the alternative between “stabilizing” flakes as is and getting rid of flakes is the wrong alternative to begin with.

But even if devenv has much better UX than flakes, it doesn’t cover all use-cases, or does it? Does Outputs in devenv mean any kind of output that I can have in a flake?

122

I really wished the new nix CLI outside of the nix flake command wouldn’t depend on flakes.

Flakes are imho fine. A bit rough around the edges but I still use them quite a lot. But they aren’t the only and often not the best way to do things.

123

Yes, any output you can have in a flake, but you define outputs schema yourself as a module option via types.output.

2 Likes
124

I’ll chime in just for a moment to say I see surprisingly little disagreement in this thread compared to how much communication traffic is happening here:

  • Yes flakes are used a lot, which is unsurprising given the amount of advertisement and advocacy dedicated.
  • Yes it would be disruptive to somehow “kill flakes”, as it would be disruptive to abruptly discontinue any sort of legacy tool or interface. We shouldn’t do that
  • Yes there doesn’t seem to by any need whatsoever for flakes to live in “The Nix” code base. It’s always seemed peculiar that flakes aren’t just an external tool, and it seems the weirder now that there’s so many alternative implementations of Nix which could all be used as “backends” for such a tool…

My hindsight opinion regarding flakes (I discovered Nix much later when they were already a thing) is that I think we do have technical problems that flakes could have solved but they haven’t, and I think there is a UX problem that they do solve, but solve poorly, and the way they do they should not have required a single extra line of C++ in “The Nix” implementation.

Specifically, I think the two technical reasons we need something flakes-like are:

  • A dependency injection system for composing out-of-tree projects without creating combinatorial explosions (addressed by inputs.<name>.follows but poorly)
  • A user interface for pure/hermetic evaluation

The UX issue is merely the lack of an established protocol: so a project (say disko or SNM) exposes a bunch of nixos modules, how do you figure out where are the entry points. This is simply a matter of convention: ok our top-level attrset will have this map nixosModules, and attributes under that would be of type you can include in imports. This doesn’t require any special features in Nix the store, the interpreter…

Technical issues flakes do not address (but maybe the misleading documentation convinces people they do):

  • Exposing packages or package sets. At least this is my interpretation of tomberek’s FOSDEM talk on “units of compositionality”
  • Composing out-of-tree projects in a configurable way. Cf. e.g. the “nixpkgs config should be a flake input” issue

Issues that are addressed by something other than flakes, but flakes take the credit:

  • CLI UX. The nix3-command really is an improvement over the previous generation. I’ve taken on to use nix build -f "<nixpkgs>" ... over nix-build or over nix build nixpkgs#.... Nix3-path-info is much less confusing than nix-store (still not without flaws). Note that nix build -f "<nixpkgs>" still supports configuration using --arg. It’s mostly profit and virtually no loss? I suppose the nix-command wasn’t stabilized just because the implementation was entangled with with flakes? And it still does feel odd that it’s part of “The Nix” code base rather than external, but ah there’s always nits to pick.

I find it very ironic that this thread is comparing detsys Nix to devenv: an external CLI tool that wraps Nix[^1], and in case of the “Determinate Nix” an external daemon that “drives” it. That’s what flakes could’ve been, and maybe still can. That’s how we could integrate a more complex build scheduler too: by making it external…

Trying DetSys Nix, devenv

I apologize and I don’t mean to be mean but I have to say: Discord, lol :upside_down_face:

EDIT1:

[^1]: I hope I’m not misrepresenting devenv? I’m not closely familiar

EDIT2: Maybe worth mentioning that I do think there’s potential benefit to implementing lock-files outside nix-lang, and specifically in an external “dependency injection” system for reasons similar to those concerning evalModules, but this is a discussion for a separate thread

7 Likes
125

Why do we need to copy everything to the /nix/store to maintain isolation

This is my biggest grief with flakes. Its very slow and the purity argument does not justify it with the current performance

3 Likes
126

If I am allowed to speak without getting flagged (been a while)

fwiw, that is exactly what we are going for in Ekala starting with the cornerstone, a flake replacement that actually solves the problem (provisionsally called atoms). Nix is a novel thing, it stands to reason that a somewhat novel format may be required to suite it’s ideosyncracies. I believe atom’s are this, but I’ve got a bit more work to do before being able to effectively communicate why I think so (without overloading this thread in detail, at least)

With that said, I am quite done “discussing” this non-sense again and again in circles, keep using flakes for now, assuming I reach my goals like I hope to, there will be little to no reason to do so long term, it is of little consequence (to me at least) in the interim.

P.S. My opinions don’t exactly feel welcome here, I think every post I’ve made in this discourse for the last 6 months or more has been “flagged”. You guys can just ban me you know? Until you do, I’ll just keep speaking my thoughts uncensored, because that’'s the sort of communication that will be required to actually solve these problems (and the reason, imho, why we keep going around in circles)

8 Likes
127

Flags generally come from the community, then a moderator gets a report and can remove or agree with the flag. The mods of this forum have never been particularly quick about acting on flags (I’m sure there are a ton of them), so it is your peers here that are getting your posts hidden, not the moderators. Your conflation of the flag and the ban is incorrect.

2 Likes
128

Flags generally come from the community

No, they actually come from specific people. Not from some anonymous collective. I never used the flagging feature so far.

Back to topic: I think it’s really a good sign that more experimentation starts to happen in all sorts of direction. And hopefully the new steering committee will have the courage to act on the technical situation regarding flakes.

In a way it is unfortunate that DetSys hasn’t waited with all these controversial announcements until this situation is resolved and there is actually a body of people elected and empowered to provide technical leadership.

4 Likes
129

Yes that is how the flagging feature works.

The person must have an account on this discourse instance, which is one of the main places the nix community gathers. So certainly not some “anonymous collective” or whatever you mean by that comment.

That’s your choice; obviously other community members feel differently.

1 Like
130

This post was flagged by the community and is temporarily hidden.

6 Likes
131

Is there a roadmap for ekala? Briefly looked into it and I’m curious to try it out.