Improving flakes

Why?

I’m starting to sense that the community is potentially at risk of one day fracturing into two distinct factions the longer that flakes linger as an unstable novelty. On the one hand, flakes offer an integrated approach to a problem that many people have tried to solve using quite a few external tools introduced throughout the years. For those who have jumped on the bandwagon, flakes, while far from perfect, seem like the proper approach, at least in theory.

After all, why should we delegate revision management and modularization to an external tool when this is really a core language problem? For many, myself included, using an external tool just feels like the wrong way to do things, and we’ve taken up an active role in evangelizing flake usage, if for no other reason than to flesh out their problems and bring a stronger final product to a stable release.

On the other hand, it’s clear there are some warts that make using flakes quite unattractive for long timer nixers, and their complaints are far from irrelevant. Many respected members of the community seem to have taken up quite a negative view on flakes, and their stance seems justified in many respects.

Rather than letting the division fester and grow into something more sinister, why not attempt to bring these two viewpoints together and find a compelling solution that addresses the problem of flakes head on?

How?

A nice end goal for this thread would be some form of draft RFC to fix what currently urks the community about flakes so we can finally have a proper solution to the problems that they try to address. The faster we can get a solution into a stable release, the less chance we have of something like a hard fork dividing the community.

Of course solving what’s wrong with flakes isn’t a trivial problem, and that’s probably got something to do with why many of us have put it off until now. I’ll try to kick things off with a few of the issues I’ve experienced with flakes since they were first introduced.

I understand time is precious, and especially for those not to invested in flakes, it may not seem worth the effort, but voicing your opinion here will give us more insight into how to properly address this going forward.

What?

I’ll start with a list of a few things I really feel need to be improved before a stable release is cut:

  • While flakes work fairly nicely at the top-level of a project, they really fall short when attempting to use them from inside the project as a sort of module layer. If you put in enough (quite a bit of effort really), you can actually do some pretty cool things with subflakes, like modularizing your dependency structure, i.e. inputs. This allows a cleaner solution to the miriad of fetch* functions in nixpkgs, as one can simply point the src of a derivation to an input.

    However, the interface is anything but intuitive at the moment. One must continually specifying flake = false for every input that you wish to use as a package source. I’m starting to think that maybe we should have a separate notion entirely to complement flakes. Something that is very similar to a flake.nix but isn’t directly accessible to the outside world the way subflakes are currently.

    Maybe there could even be a few different types of interal subflakes (or modules if you prefer), all hidden from the outside world by default and cleanly referencable from the top-level flake.nix. A sources.nix for packages sources, a mod.nix for interal library functions, a pkgs.nix for logically related package definitions, etc. Each one should be known to Nix in advance and easily referenced in a flake url. Maybe you could name the file whatever you want, and specify it’s module type in the flake uri itself, e.g. "pkgs:some/path/to/myPkgsModule.nix".

    Solving this problem nicely might put the argument to rest that flakes only make sense if we decentralize nixpkgs. A solution of this kind would allow us to keep the monorepo in tact, while having a clear and idiomatic modularization paradigm.

  • Another annoyance I’ve run into is the restrictions on specifying inputs themselves. I realize these restrictions are in place for a very good reason, i.e. to prevent impurity from leaking in. But really, all I want to do is specify inputs in a less verbose way. One solution might be to allow inputs to be optionally specified in a list, or imported from a inputs.nix or inputs.toml file in the toplevel that only contains a list of flake uris. So instead of inputs.someInput.url = "git:some/git/ref";, it could be inputs = [ "git:some/git/ref" "github:owner/repo/branch" ];.

    It seems like a missed opportunity to go to the trouble of defining a nice uri spec for flakes, only to require such a verbose declaration before you can actually use it. Combined with the different module types mentioned in the previous bullet point, we could avoid having to repetively specify things like flake = false by giving sane defaults to inputs of a given type.

I’ll leave it at that for now, as this has taken longer than I initially planned and I’m gettng pretty tired, but I hope we can all come together on this and produce a final product we can all be glad to use. I may update this list later with some additional insights, but hopefully it’s enough to get the conversation started.

12 Likes
  • An additional point is that flakes, unlike some of their language cousins like e.g. pyproject.toml do not make a distinction between development dependencies and build / runtime dependencies. This leads to the unfortunate situation where a mere chore dependency (e.g. on numtide/devshell) injects itself into the downstream dependency tree, unnecesarily. I’m not sure how much of an issue this is, but I’ve noticed it as a conceptual shortcoming that has come up in more involved discussions.

  • Further, flakes handle inputs by version controlled commit hashes, but at the same time do not (yet) have a builtin notion of patching. This cuts their vcs-enabled composability just short of the true power of patching. While this can be worked around, this is unergonomical, since in the particular fast-moving environment in which flakes are applied, patching is oftentimes a prime necessity until such patches are upstreamed proper.

  • Thirdly, flakes operate two notions of reusability: the “pinned” version in the form of self.packages and the “unpinned” version in the form of either self.overlays or self.packages combined with the caller making use of inputs.follows. Both notions optimize for different scenarios. But the fact that they are so entwined — with the unpinned notion even exhibiting a “double” API — makes them harder to apropriately reason about. In adition, the current flakes specification does not hint nor enforce any version boundaries for the unpinned notion. This might lead to hard to fix and subtle incompatibilities, especially in deeper nested dependency trees, that are entirely left for the end user to discover. I don’t really know a good answer to this aspect, but I still wanted to bring it up for discussion.

I love flakes and want them to shine like :sparkles: . Is a rename out of scope? :wink:

11 Likes

I am quite neutral wrt. flakes. I think they offer some nice benefits, such as: a standard interface for Nix repositories and much better enforcement of purity. However, there are some things that I find clunky:

  • Inputs url and follows are stringly-typed.
  • That package sets are specified per platform. I understand why this is done, but the fact that almost everyone is using flake-utils shows that it is suboptimal. @domenkozar proposes to make system a flake input rather than output: Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.
  • Packages (outside legacyPackages) can only be one level deep. This may be ok for package sets, but is annoying for some particular use cases. E.g. I have a Nix repo with programs + models (as in machine-learned models). It’s really nice to have some structure such as models.<modelName>.{wrapper,docker}.
12 Likes

I wrote on and on about how I’m dismayed about flakes and the lack of roadmap and my inability to evangelize Nix in the meantime, but it feels a bit off-topic from this focus on specific issues. If I could have one present for the whole year, it would literally just be a roadmap with a loose timeline for flakes in Nix.

Anyway,

On the other hand, it’s clear there are some warts that make using flakes quite unattractive for long timer nixers, and their complaints are far from irrelevant. Many respected members of the community seem to have taken up quite a negative view on flakes, and their stance seems justified in many respects.

What are these? (Edit, to clarify, that’s not direct at anyone in particular, nor am I asking folks to speak on someone else’s behalf. It’s just that I also have gotten this impression, but my understanding of it is somewhat nebulous and I’m eager to understand better in the hopes of there being a best path forward.)

9 Likes

I wrote on and on about how I’m dismayed about flakes and the lack of roadmap and my inability to evangelize Nix in the meantime, but it feels a bit off-topic from this focus on specific issues. If I could have one present for the whole year, it would literally just be a roadmap with a loose timeline for flakes in Nix.

I don’t think this is misplaced at all here. In fact, that’s my primary concern with flakes. I really like that feature, I regularly contribute code and use it for personal stuff. But I still think it should not be advertised for production use or to newcomers as soon as it’s unstable:

  • In (the now-closed) RFC49 there was a discussion about whether Nix is the best choice for flake configuration or if e.g. TOML is better (just like there’s a Cargo.toml in basically every Rust project). It was suggested to re-consider that in later flake iterations. Since I’m not aware of a statement from e.g. @edolstra this basically means that we’re all at risk of having to rewrite our flake.nix expressions at some point.
  • The fact that there’s no clear roadmap also leaves me with the (of course subjective) impression that there’s no actual plan for a release, but instead a lot of new stuff is added (not just flakes, also for CAS & many internal refactorings). In fact, the diff is already larger as the one between 1.11 and 2.0:
    $ gd upstream/1.11-maintenance  upstream/2.0-maintenance |wc -l
    79329
    $ gd upstream/2.3-maintenance  master | wc -l
    114613
    
    I mean, I like flakes, I like working on improving it and all that, but I’m fairly skeptical that it will get into a stable state quickly and thus releasable.

So, this is just my personal impression, but to me it seems as there’s no plan - or at least no transparently communicated plan - on what’ll happen next.

10 Likes

How can we get @edolstra to weigh in on the subject of a roadmap for flakes?

2 Likes

What are these? (Edit, to clarify, that’s not direct at anyone in particular, nor am I asking folks to speak on someone else’s behalf. It’s just that I also have gotten this impression, but my understanding of it is somewhat nebulous and I’m eager to understand better in the hopes of there being a best path forward.)

Well I left this part out explicitly, so as not to call anyone out. I’ve heard various comments ranging in depth from “I hate flakes” to more involved arguments about how they will cause a split in nixpkgs, which is probably the concern I’ve heard echoed the most. I was hoping to encourage those with strong opinions to weigh in themselves, rather than attempting to speak on their behalf.

There are also a lot of warts, like the few I mentioned up top, and I’ve got a few more I’ll probably add later on. It looks like the lack of a clear roadmap is a big one for many people, myself included. I was focused more on technical shortcomings in my OP so I’m glad you brought this up as it’s a really important point.

3 Likes

You mean a package source (repo) which hasn’t a flake.nix file i suppose?

I’ve had a discussion about Flakes with a few people in private.

To maximize the chances of Flake feedback be taken into the account, I’d really suggest splitting the discussion to one specific aspect/point. If everything is brought up in one issue, it speaks about the Flakes in general instead of limiting the discussion to a specific issue.

Writing the system input proposal and talking to people it feels to me that the underlying issue about Flakes is that flakes are the first attempt of “convention over configuration”, but only in the aspect of the CLI, but not also the specification of the Nix file.

That gives Flakes a huge design space, that each of us will find a different way, as when things get opinionated, there are lots of opinions.

My take would be: let’s get Nix 2.4 release out and make another iteration of the design in the upcoming months. This will provide feedback from even more users and allowing us to make another round of improvements. It’s a lot better to learn from current design now, rather than hold on to it forever (sunk cost fallacy).

The fact that we are reluctant to advise the use of flakes is by design, it’s marked as experimental and still unreleased. What would help is to define what experimental flag means so that expectations can be held accordingly.

13 Likes

Preface: I like some of the changes that the current Nix unstable development provides but I fear the whole changes that “we” are trying to get into 2.4 are too big already. We’ve had several parts of the code changed in drastic ways. The bug budget for a stable release has IMHO been exhausted a while ago.

I had these comments for a while already but only ever brought them up in many private conversations but never put it out there in “public”.

Here it goes:

I for one think we should circle back a bit on what flakes are supposed to fix and identify issues with the current solutions (random default.nix files…) and what approaches we can take to make the situation better. As Domen pointed out this also means defining what an experimental feature is.

A first step to agreeing on a proper Flakes implementation is to decide what it should solve. Solving “all of the problems” at once will probably not lead to an ideal situation. What are those problems anyway? Once there is an agreement on what it is we want to solve we can start experimenting with solutions and incrementally make the world a better place.

The whole RFC did go sideways and didn’t have a clear direction. The implementation changed as the RFC was going forward. Changing code while discussion is ongoing causes lots of churn on those involved. This is equally true for those trying to follow the RFC as well as those brave souls implementing it.

Flakes, as implemented right now, is trying to do many things but it doesn’t feel well rounded for usage from beginner to advanced scenarios. What was the goal here? Is a huge monorepo (with sub(-sub*)-flakes) a valid use case? Is anyone not using any of the support libraries that are required to write half-maintainable flake.nix files for more than just x86_64-linux? What does it actually include?

Since defining and implementing a standardized file format for flakes isn’t enough there is also the whole side-quest of overhauling the CLI. I am aware that flakes - in its current form - want to be as hermetic as possible and thus sets a few evaluator options to make it so. For an experimental initial implementation we might as well implement flake build as a wrapper around nix-build flake.nix --option ... -A .... until the file format and the general direction of flakes are decided and agreed upon.

In order to properly iterate on the Flakes implementation quickly (with everyone in the community having a chance) it would have been much better to start out by implementing Flakes logic in pure Nix instead of the Nix C++ codebase.

Nothing would have been wrong with import <nix/experiements/flakesN.nix> { inputs = { ... }; outputs = { ... }; until we are happy with UI & UX of flakes. For reasons of efficiency and future UX improvements it could then be moved into the Nix C++ code - if required at all. Before the actual implementation is “stable” we could have bundled experimental flake files with Nix in point releases. Importing those would also “unsupported” and should throw a warning on every single use.

Features such as lockfiles can and could be implemented as well. We already have command line tooling to generate “lockfiles” (nix-prefetch-git .. | jq .. > lock.json). Obviously in the end the tooling should be integrated with the Nix installation.

When iterating - locally - on the flakes format all you would have to do is exchange that path that the flake definition is imported from. Currently you must be fluent in C++ and understand all the indirections the code, the fetchers, the new multi-inheritance CLI classes, … are taking. At this point it should be obvious that contributing non-trivial changes to Nix requires a lot of dedication whilst we can see in Nixpkgs that we have plenty of people that are willing to write Nix.

Perhaps the lockfiles story should have been tackled after the initial version anyway. Nix users are used to pin their dependencies explicitly. Pinning dependencies also relates to the whole story around Nix having it’s own (flake) registry. Who does it serve? Does it just safe us the github:nixos/ prefix when referring to nixpkgs? Is that really worth the feature? There was a whole sub-discussion on the RFC that was not taken seriously as it was officially not part of it but yet is an essential part of how flakes fetches inputs. What is gained by introducing yet another notation for declaring inputs? Couldn’t we fix whatever was wrong with the fetchers? What is gained by having the inputs be an attribute set instead of actual Nix code? Why do we have to take them into the inputs = { ... }: function instead of using regular Nix code that used to be good enough? I feel the overall community doesn’t know why that was done or even if it is a good way it is now. It certainly serves the new CLI.

As a general note I am impressed on how well the Rust community does this. There have been RFCs to define a general direction of a feature and then there is follow-up work to actually flesh out the implementation AFTER the direction is clear. What happened here is nothing like that. We just get an RFC while we discuss an already existing implementation just to amend the implementation.

20 Likes

I definitely agree that it can be overwhelming and somewhat unproductive to discuss all the issues with the current implementation all at once. But at some point we’ve got to distill these issues down into some form of RFC so, as @andir mentioned very eloquently, we can have a clear direction for the devs to move toward.

I myself have been very frustrated with my own person lack of C++ experience, and have tried and mostly failed to address some outstanding issues. The best I can hope for right now is to push the conversation forward and I’d be more than happy to help draft an RFC to bring everything together once we work out a clear direction for flakes.

It has been difficult to discern where best to try and address these issues since not all of the core Nix devs seem to frequent discourse, but I thought it might be a good place to start. The current situation of flakes being unstable for several years seems to have caused a lot of confusion, exhaustion and frustration for all parties, and I’d really love to see some concrete steps taken to finally move towards a merged release.

Even if the ultimate decision is simply to completely scrap flakes and move forward with a completely different solution, I’d like to see that codified as soon as possible.

After all, the longer we wait, the bigger that diff is going to grow…

6 Likes

So this is going to be from the pov of someone that at this point consider a hard fork like the only way forward. I have zero hope of this conversation getting anywhere sadly.

I think part of the problem is that the nix community has 2 different type of users. One one hand you get the people that seem mostly to come from the Haskell side of the community. What they want is to sandbox better, deal with impurity and limit the reproducibility problems.

On the other hand, you have people coming from a more ops or other dev community pov who mostly want a nice way to dive a dev, build and deploy environment. Something that is particularly hard to get out there outside of Dockerfiles, which has all kind of problems.

This second population has yet to understand any advantage of flakes. This is a population that is generally really happy about the 2nix, FOD or fetch stuff and that mostly see flakes as a way to make them all kind of work that would not be sustainable. How much of that is true is left as an exercise to the reader. It also a community that mostly do not want to use Git at a distribution format.

When reading the Flakes proposal and the development since then, the feeling of that second part is mostly that they are ignored. Is that true ? Maybe not but it is the feeling.

This is not helped by the fact that Flakes were presented as a “fait accompli”, with an implementation already in the core repository, with a fully fledged implementation. Especially knowing how hard it is to approach and contribute to the Nix codebase itself. This felt like a “this is how things are going to be, deal with it”.

I don’t know what steps can be taken from there. I would advise to do a build flag next time instead of a runtime flag to run experimental. Do not make people feel like this is a decided way and we are now just stabilising, as it is how it felt this time.

I do think that Nix will have to decide what they want to be on the long run. The community as it is right now is too diverse with too diverse needs. Unless major work and money is poured into building a codebase that can deal with them all, hard forks and explosion into more granular communities is probably a better way forward. The main reason that has not happened yet is mostly that the Nix codebase is considered mostly intractable.

3 Likes

I suck. I’m embarrassed to show my face in the community because I have been a ‘ghost’ maintainer (might still be listed in Nixpkgs, idk), and I used to make minor contributions every now and then, and I don’t anymore even though I still use NixOS as my main desktop OS, I use it at work, and I follow developments in the community closely. Every criticism I have related to Nix is directed, one way or another, at people who give more to the community, have more expertise, and are more responsible than I am, which makes me want to keep my thoughts on a lot of things to myself. (I’d also like to note, like I think others have in this thread or similar threads, that the situation with Nix flakes is so painful in part precisely because the existing implementation really does solve a bunch of problems and facilitate valuable new ways to use Nix.)

But I’ve been thinking about this a lot lately, I wanna get it out, and I think some of what I have to say might be useful.

When I first got into Nix, I enthusiastically Nixified my setup, creating a config repo that I used to take my tools with me on NixOS, foreign Linux, and macOS. I used git submodules for checkouts of Nixpkgs, which essentially worked to pin my setup, but it was/is cumbersome. For a hell of a long time now, I’ve told myself that it’s time to get back into Nix, starting by rewriting my Nix configs in order to simplify them, so that I can pin Nixpkgs and other sources of Nix expressions in a more lightweight way, and so that I can work on NixOS modules without keeping a fork of Nixpkgs around just to test and develop them. After that I’d like to fix some things I broke ages ago, if they’re still broken, and work on some new configuration stuff for fish and other shells. I’d like to be able to conveniently do the latter outside of the Nixpkgs tree because I’d like for others to be able to use it regardless of whether I ever try to get it merged into Nixpkgs.

Every time I’ve looked into it, I’ve been put off by the state of flakes and other available options for pinning the local system on NixOS. For my personal use, I hardly even care about the flakes interfaces (the one for Nix code or the CLI) being stable, but it’s still never really looked like a good option for me. The problems have instead been as follows.

  1. The Nix ecosystem involves a ton of tooling that’s more or less third-party, or independent. Because the flakes feature is considered unstable/experimental and might disappear, support for flakes has lagged in many of these tools, and has arrived at various times. On different occasions when I looked into switching, different tools that were part of my setup or that I wanted to be able to use, like NixOps or Home Manager, didn’t support flakes. I looked into branches and forks where support had been added, but it was more steps and I didn’t want to deal with it. This kind of thing is still a problem for some tools.

  2. Actual flake.nix files for NixOS configurations are complicated. Besides boilerplate syntax (tons of flake = false;) there’s also often some logic involved in them many of them. This is the equivalent of a general purpose package manager’s sources list, and many configurations I might want to model mine off of include bits of code for things like:
    • dealing with all the different target systems in the way the flakes interface forces you to
    • managing NIX_PATH or some kind of backwards compatibility
    • patching Nixpkgs
    • emulating the Nixpkgs flake by setting up legacyPackages so that the convenient nix profile install syntax works
    • setting up overlays from various pinned package sources
    • configuring Hydra jobs or other CI

(I guess some of this stuff is what Nix flakes are for, and I can see why it makes sense to put that stuff in flake.nix. But truthfully, I mostly just wanna pin Nixpkgs and have it work with nicely with nixos-rebuild, and the best examples I have to look at are all over 100 lines long (often several hundred lines!), and it’s not unusual for them to pull in whole libraries like flake-utils or flake-utils-plus in order to be a little neater. I want to get started using flakes to pin the existing dependencies for my Nix setups (NixOS, macOS, foreign Linux) in like 20 minutes or less without dumping a bunch of helper functions in my flake.nix or building from a template whose conventions may be motivated than much more complicated setups than mine or an assumption of NixOS-only. There’s still no obvious starting point for that, and it reinforces my procrastination/hesitancy.)

  1. The Nix flake registry doesn’t have good backwards compatibility with Nix channels and NIX_PATH. It feels incomplete and inconvenient to use until (a) the Nix registry presents its entries as pseudo-channels for tools or commands that still expect NIX_PATH, and (b) there’s some way to configure a systemwide/multi-user Nix registry on non-NixOS (or imperatively, via the command-line), like you can using root's channels.

There are other issues, but those are the reasons I’m not using flakes right now, even though it’s clear to me that they solve important problems and I’d basically like to be using them already.

I think some of the ‘extra’ stuff flakes do is interesting and might be necessary, but if nixos-rebuild and nix-env worked as well with Niv as they do with nix-channel, I’d just start using Niv for my NixOS configurations tomorrow.

So that’s what’s holding me back or turning me off when it comes to flakes, but the limbo state of flakes (and the other new features in nixUnstable) also puts me off recommending Nix to others. :frowning:

You only get so much interest and patience when you’re trying to get someone to try new things, and I don’t want to waste the one chance I might have with someone recommending a version of Nix which is only almost there. Here’s what I mean:

  1. Nix is making its way into the repositories of package managers for other distros (it’s already in Debian, Gentoo, and FreeBSD Ports, while it’s on the way in NetBSD’s pkgsrc). This is awesome, because it means trying Nix can be easier, safer and more manageable. It could put it in a similar position to Flatpak or Snap for users who want to get versions of packages or whole packages that aren’t available in their base distribution, where its presence in the base distribution makes it feel like a natural addition rather than grafted on, the way a glorified curl|bash installer can seem. But, for example, the Debian package doesn’t set up NIX_PATH the same way as NixOS or our tarball installer. But why deal with holding someone’s hand in figuring out what their NIX_PATH is and where it comes from, especially when I know that ‘any day now’ Nix won’t depend on NIX_PATH anyway? It also doesn’t add any channels by default at all, so even setting aside the impurity, nix-env won’t know how to install anything out of the box. Best, it seems, to just wait a bit before recommending Ubuntu users apt install nix-bin nix-systemd-setup.

  2. Same thing for recommending NixOS for local desktops or simple server setups when the selling point is reproducibility and the best tool we have for pinning NixOS setups is unreleased/unsupported and kinda incomplete. I want to wait at least until we have something like devos a little more settled so that I can tell someone: ‘here’s how you set up NixOS and pin it to a commit, here’s where you put the config for each host, here’s where you put any custom modules you want, here’s where you put custom packages— and you can start thinking about how that stuff works and whether you want to rearrange it as you become comfortable’, knowing that the starting point for them is simple and unlikely to change from what I use or know in the near future.

  3. Similarly, all of the CLI improvements in nixUnstable, the fact that those will still change, and the massive performance difference yielded by flakes thanks to the evaluation caching it supports all make me want to put off encouraging people from really trying and learning the Nix CLI until after that stuff is released. It also doesn’t seem great to just introduce them to the existing/old CLI when I know that something much better is fairly well developed, mostly documented, and fairly widely used. Some of this is stuff that the community can work out independent of the first stable release of Nix that enables flakes, as seems to be going on inside devos and flake-utils-plus. But to some extent, projects like this that aim to simplify or settle conventions for the use of flakes inherit this problem of volatility (likelihood to undergo breaking changes in the near or moderate term) from the flakes feature itself.

Overall, the sense that later (but soon) will be a MUCH better time to recommend Nix seems like a real problem to me. And it’s a problem that might stand in the way of the development of future Nix goodies. You can even imagine something like a deflationary spiral except the investment being disincentivized is developer time in the Nix ecosystem. Perhaps this worry is overstated. Lots of things are definitely still moving forward in the Nix ecosystem and it’s exciting to see. But I can’t help but wonder if we’re missing out because of this effect.

I think maybe I’d rather have a flakes interface that’s settled and sucks, or see a promise of stability actually get broken, than see Nix spend another year or two in almost-there limbo for the basic functionality of pinning package sources and incorporating third-party repositories in a way that every tool in the Nix universe is expected to support. If we have to start out with an initial version of flakes that’s kinda clunky, I know I’ll figure out a way to deal with it and I bet the rest of the community will too. But the expectation that this implementation could disappear or radically change some time soon makes me (and, I assume, others) hesitate to bother really figuring out how to do that now. :-\


edited to make it a little clearer, more readable, and more polite (no important differences)

10 Likes

Generally I’ve liked having flakes as an opinionated structure. I’ve eschewed the wrapper utilities and have used them to pin, to create Hydra jobs, NixOS configurations, etc.

I’m not sure how to address all of the problems people have discussed, I only know how to solve problems: one at a time. So if people want to work with flakes, get something working, or even to help me understand what difficulties people are having, please put an hour or two in my office hours.

4 Likes

As a mostly passive observer and Nix/OS user for more than 2 years who has just started contributing to nixpkgs, I am barely at a stage to have a fully informed opinion, but thanks to @nrdxp we are having a much needed discussion on this.

Personally, I feel Flakes are starting to hold Nix back.

  • We have bunch of nice improvements in Nix unstable, including new CLI and CAS. Some of these are entangled with Flakes, and are holding back wider adoption because flakes are so underbaked.
  • I remember being excited about flakes way back after watching NixCon session by Eelco. But then the RFC went nowhere. This has many issues. I wanted to try out flakes immediately, but the documentation was so poor that for a Nix newbie it was too much. I can already see the rift happening, because nix already has sad documentation story and even then ongoing efforts are split between flakes and legacy way.
  • Latest NixCon brought possible future of TOML flakes, which means whatever work I have put in, may get deprecated.
  • I agree on the point that RFC did not discuss and pan out implementation details. we got an implementation and then there was discussion on how it could be take forward.
  • We still lack any kind of transparency on what the future looks like, and from the history of this feature, even if we get one, I’m afraid it will be like RFC where we’ll be told what side we are already on.

The list may look like its all over the place and its partially my fault for being bad at communication, but here is what I would personally like to see.

  • Flakes solve real problems, lets first decide what they are and in what order they should be solved. In short, scope definition.
  • A fully open discussion on the above.
  • A timeline on the above. It can stretch for 5 years and will be understandable because such changes take time.
  • Consistent releases that solve individual parts from above. Let the small feature permeate slowly, so the community gets time to adjust and test and hash out finishing touches going forward.

I fell in love with Nix/OS the first time I tried it. I hope to continue using it, and I hope to be able to use it professionally some day. But as it stands, its already niche, and further ongoing lack of clarity is really going to hurt long term. Let this not be Python3 moment for us.

12 Likes

I’m closing this thread to encourage people to split their improvement points into single issues like @domenkozar asked. If you have concerns about this, please direct message me.

3 Likes

I received a bunch of feedback that I shouldn’t have closed this, so I’m reopening it.

8 Likes

Some notes from office hours:

  • builtins.getFlake is not well-known. With a new enough Nix it allows non-flake uses to find and refer to flake-enabled packages/overlays/etc.
  • recommend not using flake-utils or other “flake wrappers” until after one is comfortable with making a few explicit flakes
  • it helps to at first only use flakes with devShell = import ./shell.nix {inherit pkgs;}; and defaultPackage = import ./default.nix {inherit pkgs;} as a way to ease adoption and support both legacy and new (once comfortable one can adopt flake-compat)
  • avoid nix profile (vice nix-env) until one is comfortable
  • people are used to using nix-shell for both build-time debugging, quick test of programs, and development shell. It helps to clearly split these consideration; flakes help here, though it can be non-obvious. (nix develop and nix shell)
  • apps != packages
6 Likes