The data on how people discovered Nix and comparing that to years of experience was especially interesting and elegantly presented.
I was surprised to see such a high level of Lix usage already. Congrats to the Lix team for quickly building interest and confidence in their fork!
It stood out a lot, too, how virtually everyone who has been using Nix for a while eventually picks up NixOS.
One nitpick: I’m much too colorblind for the visualization of the data about which resources people rely upon when seeking help. Totally inscrutable to me. An old-fashioned nested list with percentages and response counts might be a nice complement.
Well done! Overall really well-presented and I think the data collected has been more useful and interesting than ever.
Nix 2.18 was the default in Nixpkgs and NixOS during the survey, so I would expect the vast majority to have run that version.
2.19-2.21 did see fallout from our work on fixing fetchTree, part of its stabilization, and therefore part of flakes stabilization. We can’t commit to hash-for-hash stability when fetching has impurities and bad defaults. Also we’re working towards an actually deterministic version of lazy-trees, as performance is a significant issue in Flakes.
This is a reason why one of the (turned out, later) Lix maintainers held back the Nix version in Nixpkgs.
I don’t want to go into the legitimacy of that, but having 2.18 as a baseline for Lix comparisons benefits that project. I’ll leave it at that.
NixOS 24.11 is scheduled to ship Nix 2.24 as its default, and you’re right that - if/when found - handling any new regressions must be high priority.
Just for reference: one of the main reasons Lix was started was because a few of us wanted ato be able to move to a newer Nix, but were blocked by regressions. I agree there’s a causal link there, but think the cause and effect are flipped from what’s implied here.
The Nix team could have used some help with those regressions back then, but received very little.
It makes me sad, whichever way the causal link is. I don’t think I can know. Probably both? I’ll leave it at that, because I’d prefer to have a forward looking discussion if we’re going to have one, but let’s stick to the topic of this thread for now. What I wanted to point out is that some people’s perceptions of our respective progress-es are biased because of this situation. I hope it’s ok to identify such a bias.
I feel like there’s some minor memory-holing of where we were in, say, 2020.
About 2 years passed between the 2019 release of 2.3 and 2021 release of 2.4–and the change to a 6-week release cadence it marked was a compromise on the previous release process in recognition of the problems caused by going that long without a release.
It was pretty obvious that arbitrarily moving to a 6 week release cadence would cause its own problems. There were obviously issues deterring release, but I would argue it’s fairly likely that having to adapt to that release cadence is and will continue to pay dividends for the project.
(FWIW, I believe the “nix team” formed in 2022.)
I don’t think anyone is or should be happy with the number of releases that haven’t been nixpkgs-ready, but about 13 months have passed since the first 2.18 release, which is still a good bit better than ~24.
I forgot to respond to the survey (yes, lmao) so maybe there was some clarification there I’m unaware of, but I would be unsure what to pick here exactly. I use NixOS (well, at least on my personal machines) and use Nix to develop software (where feasible), so I kind of “install” software via those - but given you list “installing via nixos configuration” as something this option doesn’t represent, then what does it represent? Using nix-env (which I think we would’ve been rather unhappy with instead, as it seems universally discouraged)? Using nix profile explicitly (honestly never used them by hand)? Does running a nix shell to have a transient shell with a package count or not? If all this was clarified in the survey then sorry for the noise, but if not, then maybe a better option is needed?
Amazing work with analysing and presenting the data, @Arsleust. I agree with @pxc this is the most informative survey so far, and I’m really looking forward to the next one. I learned a lot about what we need to optimize for and how to reduce the effort of preparing and evaluating such a survey, and I hope we’ll manage to implement all of the improvements people were suggesting.
I take great issue with this statement and I think it’s fallacious. You cannot deduce that the current implementation of flakes are good or desirable based on the popularity of its feature flag.
I am one of the data points in this survey that have the flakes experimental feature enabled. I don’t have it enabled because I like flakes or even use them to any real capacity to define my environments and such but because there is no viable way to use the primary nix3 commands without them.
You cannot, for instance, use nix shell without flakes. This is an arbitrary limitation, not a technical one: Nix 2.3 had an equivalent (then called nix run) which functioned using NIX_PATH, without any flakes involved.
For my setup, I’ve set up the glorified global namespaceflake registry so that its nixpkgs points at a nixpkgs checkout in the filesystem that I manage separately. The NIX_PATH’s nixpkgs= also points at this locaiton the for the legacy commands to consume. In order to get this to work, I had to research how to make nix not do dumb flake shit like copying the entire checkout to the Nix store every single time I run a nix command which is somehow still an issue in 2024.
Would you say that I consider flakes as an integral feature that ought to sanctioned as good and stable, not requiring fundamental changes to actually function sensibly?
And yet I still have the feature flag enabled.
Secondly, while flakes are a standard to compose Nix projects, they are (to my knowledge) the only standard for this purpose. The closest things are default.nix/shell.nix which is a convention, not a standard and npins/niv which only concern themselves with fetching/locking as a replacement for nix-channel rather than exposing a project’s Nix expressions to be consumable by other Nix projects in a standardised manner.
I think it’s no surprise that people enable a feature to access a standard for which there is no viable alternative standard.
That’s like saying cars are the best mode of transport because you live in a place where transit is intentionally hamstrung and biking or walking is likely to get you killed by a car. You can be of the opinion that cars are not in any way a good or desirable mode of transport and at the same time drive a car every day because there just isn’t any viable alternative mode of transport where you live.
I think much the same applies to flakes.
“Standard for which there is no alternative is good because 100% of people who need something like the standard use it.” does not compute.
You have to show more evidence than the mere fact that people “use” it to claim that it is good.
I’m actually surprised it’s only 85%, meaning that 15% don’t use it at all. If 15% of users don’t use flakes despite being the only standard way to compose Nix projects (which I’d like to remind you that everyone must do to some degree) and are the only way to use nix3, I think that says more about flakes than the other way around.
Thank you for the feedback. I’ll be happy to better design these questions with your input. Please do join the matrix channel to facilitate this discussion.
I don’t think that’s what was said or meant at all, rather, I’d guess that they mean that removing flakes would likely be bad, since so many seem to be using them. And I’d guess further that they also hope that some of the issues with flakes can be ironed out and a push made towards removing the experimental flag.
You cannot even deduce popularity from the survey. The survey is almost certainly not representative which you can already guess from the overinflated amount of new users responding. I would expect that having flakes enabled and being a new(er) user are not statistically independent.
I’m very happy to see the interest that the survey results have gathered and that it starts many conversations!
Very neutrally, I would like to remind everyone that interpretation of survey results (and collected metrics in general) should be done very carefully. Our team tries not to go too deeply into interpretations on purpose. It will be up to individuals and the different Nix teams to work more deeply to interpret these numbers correctly. Of course, we are available to support such work.
As a general guideline, let’s keep in mind that the survey most likely has a sampling bias. We should be careful before stating that these numbers represent the entire Nix community.
It is still more useful than no data at all, as it can help filter out what to investigate more in depth.
We hope to reduce that bias in the next years by collecting responses from a wider audience. For example, we would like to add banners (prompting for participation to the survey) to as many websites as possible, from nixos.org to search.nixos.org, the wiki, etc. Also, please help us spread the word about the survey when it runs.
With that being said, let’s make the most by looking at phenomenons, trends, and investigating them further (with carefulness)!
