but it is soooo nice and simple to use! at least for any small scale project
And then it ingests all the multiple gigs of a monorepo you’re trying to use it with on every flake-related command. It’s been more than 2 years since the lazy-trees branch (Lazy trees by edolstra · Pull Request #6530 · NixOS/nix · GitHub) was opened and even longer since the issue was first brought up (Copy local flakes to the store lazily · Issue #3121 · NixOS/nix · GitHub) and yet it’s not done, while the people responsible for that PR push for flake stabilisation, instead of finishing this goddamn PR — yes, I know it’s probably not trivial, but then why not try to invest more resources into that instead of DetSys marketing? I wonder how they plan to market it to srs enterprise bznses, if flakes suck for the kind of repos such companies are likely to have? I use flakes and think they are serviceable enough for things like my NixOS configuration and whatnot, but each time I try to use it to support devens at my current $WORK it sucks. I don’t think it makes sense to stabilise something like that, without ironing those issues out first.
Before flakes you would specify the file path instead of attrpath. Both require looking at the code or otherwise running commands to inspect it.
And as a good counterexample to consistency of flakes, the schema recognises HM configs but not HM modules (prior to nix 2.24) so we have proliferation of various HM module structures.
That was my problem but I resolved it by having the flake in a subdirectory and when I do nix develop or nix build I refer to the flake explicitly and use path: reference to avoid adding any files from git.
You misunderstood the issue, the issue is lack of development inputs or ability to ignore unused inputs, so to prevent lockfile explosion people must overuse follows. Those inputs still get downloaded though… And your workaround will copy .git to the store, making it even slower while not addressing the lockfile issue.
FWIW my particular complaint is about working with big codebases - it becomes really obvious with big monorepos that are hostile to using nix, so I have to do it externally - at this point you can’t really tell flakes to ignore this or that file, like it’s going to do by default for .git and whatever’s in your .gitignore (u less I’m missing something obvious there’s no way to modify how a given output is copied. It will probably be less if a problem for non-hostile monorepos, but at some point they too will become too big to work with them. I don’t think the path suggestion would’ve worked here :75 admittedly it’s been a while since I gave up and used flakes via flake-compat, amortizing the cost on input update only.
I would be happy to learn I’m an idiot and there’s an easy solution for it, but I haven’t come across one in years.
The thread title “Nix team member suggests removing flakes” is quite hyperbolic. Most people may not realize that I’m just the documentation guy. I’m not doing the heavy lifting and I can’t make or forbid anyone to do it, either. Therefore I’m not sure what warrants dragging me into the spotlight here.
Yes, as part of the maintainer team I do have elevated influence on decisions because I tend to get involved in discussions. But ultimatly the only relevant question is: Who does the work and carries the maintenance burden?
This seems to be generally agreed-upon.
Anyone can point out problems or make wishlists. What I care about is whether we can live up to our commitments – be it as the “community”, the “project”, or the “Nix maintainer team”, each of which is often (wrongly but understandibly) perceived as monolithic by outside observers.
And from two years of experience on the team, I don’t see a way to fix the design and implementation issues with flakes in (what I think would be) reasonable time, or handle the strain incurred by everything associated in a sustainable way. I also don’t believe that the additional resources the ecosystem may gain through growing adoption will help: I fear the problems will just get more, different, and harder. I don’t want to commit to carrying part of the responsibility for the resulting disappointment.
I’m convinced it would be cheaper and faster to start over by solving the uncontroversial problems Eelco tried to address with flakes independently and at the appropriate architectural layer (which may be outside of Nix to begin with). This doesn’t require removing existing functionality, but maybe calling a feature freeze, cutting scope, and adopting a very different development and communication strategy. Part of that should be clarifying authority over the various parts of the code and the associated guarantees. All this has been discussed among Nix maintainers at length. Still, someone has to do the work.
In any case, while the flakes controversy may be the most visible stain on the ecosystem, for me it’s by far not the most important one. The overall user or contributor experience won’t magically improve by removing the experimental flag. There’s a lot to unpack about the technical realities of the Nix source code and the social realities of how it is developed. Therefore I’m not sure what else to say here.
I enjoyed reading lots of insightful comments in this thread. It’s already hard to keep track of, but I’m happy to answer further questions in isolation as time allows. Please open new threads if you want something clarified that’s not evident from public record, I’ll answer there and hope so will @roberth, @Ericson2314, @tomberek, and @edolstra.
I might not fully understand the issue but here is my setup:
Project structure:
.git/
.gitignore
src/
flake.nix
flake.lock
..
My problem was that when I ran nix develop it copied everything in my src/ into the nix store for evaluation. I didn’t want that because in my case flake provided tools and didn’t need to know about my src (which was really large and on every change it needed to reevaluate everything and copied it again and again using more and more of my precious ssd).
To fix this I change project structure to (moved flake.nix to subdirectory):
Now I can still do nix develop from root or anywhere I just need to point it to that subdirectory like this nix develop ./nix but if you do it like that you implicitly doing this nix develop git+file:./nix which will copy all files tracked by git for evaluation into the store.
To fix that I refer to the flake like this nix develop path:nix which will only copy files inside that subdirectory (flake.nix and flake.lock).
Regarding inputs.
I don’t know how to ignore unused inputs.
For all my flakes that are supposed to be used by end users I have code snippet examples in the readme which end users are expected to integrate into their config. Especially for the one flake I have which auto loads modules from a directory structure, it is not feasible for the average to advanced NixOS user to understand the code as it is quite complex.
I’m in that dataset yet I dislike flakes. I agree with Infinisil’s objections (and more), I wish we would redesign them completely. Yet I add a flake.nix to all my new projects. Why? Because what alternative is there right now? Flakes have sucked all the oxygen out of the room for any alternatives. So I begrudgingly go along with it all. But please don’t count my acquiescence as endorsement.
@grahamc you count the number of flake users as an endorsement: I think that’s survivorship bias and circular reasoning. You’re not counting:
the projects which would be using Nix if there were a better alternative to flakes, but didn’t because both default.nix and flake.nix are still too poorly designed for them
the people who use flakes because they prefer it to default.nix but actually dislike flakes and want a better alternative
The central thesis of that post is that flakes hamper the development of better solutions. In that context, counting flake usage as endorsement is not fair. What alternatives do we have?
There are some clear downsides to flakes which have been demonstrated time and time again, but they remain the only option that I think is actually worth standardizing of the ones that exist today. I actually agree that there are serious design problems with flakes, but the situation before flakes was terrible in my opinion, and users that try to use Nix for the first time wind up falling into a lot of unnecessary pain due to the amount of splitting in the ecosystem. A lot of that splitting is not going to go away until/unless flakes are considered stable. Personally, I don’t think leaning on modularity will help here, complications it would add aside, there really needs to be a module system that ships with Nix that is at least similarly advanced to flakes in my opinion. It serves an important purpose in the ecosystem.
A better design would be nice, but it’s been long enough and nothing has really emerged, and flakes themselves don’t appear to be changing substantially anymore. A couple years ago flake stabilization seemed pretty much inevitable and didn’t seem like it would even be controversial, so I am a bit disappointed that things turned out this way. I hate to be dramatic, but personally I am holding off on recommending Nix to more people until there is some reasonable resolution here, because it has become tiring to try to navigate this in a helpful way for newcomers. Having no or ad-hoc/piecemeal module management isn’t really going to cut it in my opinion.
Well, that’s my 2 cents, anyways. I’m not sure if this thread will wind up being particularly productive, but on the other hand as someone who has invested decently into flakes, the idea of flakes being removed rather than stabilized is, at worst, quite annoying. I don’t think I really disagree thoroughly on many points with people who dislike flakes on the whole, but I personally doubt there will be a perfect module system. If we have to migrate off of flakes some day in the future, then so be it, but we’re already at the point where migrating off of flakes would be hard for many of us anyways.
I’m in this camp as well. I use flakes because there’s things I do like about the feature. But there’s a lot I would have done differently if I had been the designer. I suspect this is at least a mildly common position.
There are two conversations happening around flakes in general:
are flakes good?
are flakes popular?
The original post from Grahamc is about the latter. That’s fair. He then uses that as an argument to support the former. I think that’s unfair.
This specific thread isn’t (or at least wasn’t) about discussing the actual merits of flakes; it was about their popularity, and how that is an argument in favor of stabilizing them. That argument is specious, in my opinion, and I think it’s worth pointing that out.
I actually agree that flakes are better than the previous situation. Most people do. The data shows that. The point is that “flakes or channels” is a false dichotomy; the very existence of flakes, which are a Jack of all trades but master of none, prevents the development of better alternatives. If we removed flakes and tackled the individual issues independently, we could actually come up with something even better. We wouldn’t just have to go back to channels and never try anything new again.
I think that’s what the data shows: people think flakes > channels. And that’s fair. I think that too. But that’s not an argument for “we should stabilize flakes.”
I still don’t understand why we shouldn’t stabilize flakes based on all of this, though. The way I see it, the existence and ecosystem adoption of any solution is going to be enough to discourage development of alternatives schemes. Even if an alternative scheme was produced, it would need to provide compelling enough benefits versus flakes for the existing user base to want to switch over. And while I really don’t doubt the existence of flakes has indeed discouraged some people from trying to work on alternative schemes, and I don’t deny that flakes have design flaws, I also believe a huge component of this is that a lot of problems flakes have are hard enough that it’ll be hard to make solutions that are completely better overall; there will always be trade-offs.
I think the benefit of not stabilizing flakes was that it could’ve been scrapped or substantially redesigned since it was not “stable”. However, at this point, making non-trivial breaking changes is well beyond the point of no return, and I honestly think that it would’ve been by now even without pushing from Determinate Systems et al. The story has just been more compelling for a long time, and flakes found themselves in production all over the place.
I feel bad here because it seems like flakes being stabilized is viewed as a huge loss by some, but to me it feels like an acknowledgement of a reality that has existed already. And I don’t say that to suggest that I don’t believe anyone doesn’t use flakes or to suggest that there isn’t a base of people who use it begrudgingly, but to suggest that I see the stabilization as more symbolic than an actual change in status quo. In my opinion, the experimental flags for flakes no longer serve any meaningful purpose.
Yeah, to me it’s a bit weird that these two things can’t coexist at least for a period: stabilizing flakes and working on fixing the underlying issues in maybe some other experimental feature. All the infighting and confusion hurts nix in more than one way I think.
My explanation is the fear that making flakes “stable” now, will make them even more popular to a point where it’s impossible to introduce a “flake V2” system, that’s incompatible with “flake v1”.
That’s reasonable, but I think with the problems flakes currently have, the pain will be high enough in the end to make people switch.
There’s of course to possibility to improve flakes without breaking changes, but I’d honestly prefer breaking them and make big, no-compromise changes to them and make them as good as possible. The “old flakes” should buy us some time for doing so, no?
I like how Rust’s editions feature allows backwards compatibility, even within the same project. No reason we couldn’t in the future have that with a later edition of flakes, right?
Having said that, I’m sure the steering committee will make the right decisions once it’s established. I recommend not worrying about it, if you are.
I think having proper versioning as a requirement for stabilization to allow future improvements would be sensible, however I am not sure if this can still be integrated into the current design without making backwards-incompatible changes, which at least the DetSys people strongly want to avoid.