Nix Community Survey 2023 Results

2023 Nix Survey Results

The Nix community is growing at an exciting pace, which makes it harder and more important to understand who comprises our community, what people use and care about, and the main pain points that frustrate users or impede progress and adoption. Therefore, the Nix Marketing Team conducts a yearly survey to better undestand these issues. Your feedback helps develop Nix, Nixpkgs, and NixOS to match your needs and come up with new ideas for better serving and growing the community.

tl;dr

  • Compared to last year, we had more respondents, of which a good proportion were newcomers.
  • The community may slowly trend towards more diversity in terms of technical background and occupation.
  • The Nix ecosystem is still difficult to navigate and work with (developer experience, sparse or unclear documentation, lack of recipes)
  • Nonetheless, most respondents consider that the benefits of using Nix/NixOS outweigh all the downsides.

We will try to make next year’s survey easier to complete and tailor it to better support the other community teams. Communication should also be improved to increase outreach and get more people to respond.

Total Participation, Response Rate

We received 2493 responses, an increase of 15% compared to last year.

Taking into account the fact that most questions are conditionally prompted depending on a respondent previous answers, on average questions were answered 67% of the time. Without much surprise, the simplest questions (single choice questions) had the most engagement from respondents, but the order of the questions seem to matter as well (R²=0.30).

Warning

For many questions, respondents could select multiple choices. As such, in this analysis, percentages for such questions are percentages of “Yes” answers for the choice itself, represented in green. This explains why percentages for a seemingly unique question will not add up to 100%.

Single choice question are represented in blue, and the sum of all percentages is 100%.

The Shape of the Nix Community

Respondents were either from Europe (50%) or North America (23%) for the most part.

Given that many respondents did not state their gender (14%), there is a lot of uncertainty on the exact results. That being said, it is clear that most respondents were men (>80%). We may consider discarding this question, as did Stack Overflow and other surveys.

GNU/Linux is used by a large part of the community (80%), but other operating systems like Windows (27%) or macOS (26%) are also quite widespread. Surprisingly, about 50 respondents were Windows-only users.

Most people in the community are Backend Developers (31%), which is less than last year (was 40%, meaning a drop by 25%).

Overall, the “shape” of our community is largely the same as last year. However, we do see a reduced concentration of the previously most common profiles, which have a lower percentage than last year. One could say that the distribution is “flattening” a bit, which means that the Nix user base seems to be a bit more diverse. It will be interesting in coming years to see if this is an actual trend or not.





Nix at Work

The majority of respondents who use Nix do so on a daily basis (77%). As mentioned last year, this aligns with the expectation that those most interested in providing feedback are frequent users. Keep this in mind when reviewing the data and takeaways.

Most respondents have been using Nix for less than a year (38%), which hints at an increase of new users. Unfortunately we didn’t receive as many answers from users who have been using Nix for 1-3 years as we would have expected. Indeed, last year’s respondents who were, at the time, users with less than a year of experience should now have been part of the “1-3 years” group, making it grow accordingly. This would mean that many respondents with now 1-3 years of experience did not respond to the survey this year.

57% of respondents said they use Nix for work. At the same time, respondents said that they use Nix for at least one of their personal projects (88%). This could be interpreted that Nix users are very enthusiastic and want to use Nix, but either they deem it not ready for industrial use or Nix is not part of the tech stack at their workplace.



Key Themes

Last year, we had noticed 5 key themes:

  • The “superpower”: Nix/NixOS is declarative and reproducible
  • Nix/NixOS has a steep learning curve
  • Nix/NixOS needs better documentation
  • Nix is hard to debug
  • It is hard to contribute back to the Nix ecosystem

This year is not too different: most Nix users love it for what it does well. Nix is used because it allows declarative configuration of reproducible systems and builds. Once you fix something, it stays fixed!

However, even the most enthusiastic Nix users agree on the fact that it has a very steep learning curve at every step. It is hard to get into Nix as a user, as a developer, and as a maintainer.

For each participant profile, people largely agree that the Nix documentation is one of the biggest issue. Sometimes because it is lacking, sometimes because it is unclear or not up to date. More specifically for beginners, it lacks comprehensive, accessible examples and recipes.

The barriers for wider adoption are that it is too difficult to make things work for particular use cases, or that documentation is not good enough.

Next Steps

Our goal is to help all of us understand what the community needs and wants. This information will benefit all parts of the Nix ecosystem, helping teams plan and prioritise efforts to meet the community’s needs.

We’ll be doing the Nix Community Survey again next year, and we’re working on making it even better. We gathered a lot of feedback from the last question of the survey, and plan to act on it.

I would like to thank everyone who helped make it happen yet again, and hope to see you all again next year! :heart:

Results

Below are all the quantitative results.




























Method

The Nix Marketing Team conducted an online survey of 47 questions, of which 13 were single choice questions, 17 multiple choices, 4 ranking and 13 open-ended.

The survey was open for a one month period from June 11th until July 10th. We announced the survey by way of postings to Nix Discourse, Matrix, LinkedIn and Twitter.

The survey was split into four subsections:

  • Background Information
  • Nix User Questions
  • NixOS User Questions
  • Misc. Questions

Once the survey window was closed we split the analysis into two: multiple choice questions and open-ended questions. Multiple choice was analyzed numerically by compiling the responses by count and percentages, and comparing to previous year’s results. Open-ended questions were analyzed by manually reviewing the responses. The survey analysis was conducted throughout the month of August.

Final words

Huge thanks to all the people who contributed to the survey: @kranzes @ron @fricklerhandwerk @garbas @Arsleust

If you’d like to help organize next year’s survey please contact the marketing team on Matrix: #marketing:nixos.org

See you next year!

38 Likes

A bit of unqualified feedback on the question design I’ve noticed while reading/answering:

  • Perhaps add a Nix on NixOS option to the platforms question to avoid confusion. I imagine a significant number of people might not properly read the question and miss the part excepting NixOS. If they were instead presented with “Nix on NixOS” and “Nix on other Linux distros”, there would very little room for error.
  • Use-cases: What use-case does using nix-shell/nix run for trying software without permanent installation correspond to? It’s not really “development” eventhough that’s where it says nix-shell but the tool isn’t listed under “Install software as a user” which it (IMO) actually fits in. Perhaps list nix-shell -p/nix run under “Install software as a user” aswell.
  • Ask NixOS questions first to avoid confusing the user (ideally hidden if they don’t claim to use Nix on NixOS). I’ve been confused by two instances of the survey and had to revise answers due to this. Perhaps start with a question that is unquestionably about NixOS rather than Nix first (i.e. deployment tool) to prime the user.
  • It might be a good idea to list a few very popular DIY DEs such as i3, sway or hyprland. When the most popular category with >30% is “Other”, that’s kinda weird.
14 Likes

We’ll definitely improve the situation with the Nix/NixOS sections, given some answers it must have been confusing to some.

Usually when an answer has too many responses which are “other” and it was possible to comment, we’ll see what were the most frequent responses and add them. Year after year, we should converge towards good lists of choices.

Thanks for the feedback, I really appreciate this :100:

3 Likes

Gender: Female < Other is intriguing to me, including the numbers. But I rather don’t want to discuss it, beyond just pointing it out. As noted, the root is probably hidden in the 14% who did not state their gender in a supposedly-anonymous survey (lack of trust, lack of perceived relevance?)

3 Likes

I would choose “Other” “Not Answered” myself because I don’t see gender to be relevant. Why pick gender in particular over other group categorizations (race, ethnicity, disability status, BMI, skin colour, age, height, baldness, financial status, etc.)?

I support discarding it.

2 Likes

Why not the option “Not answered” ?

EDIT: I mean, it’s fine not to answer, but “other gender” is an entirely different category in my mind, and I respect that the two genders are not a sufficient choice for everyone who wants to answer. (I don’t recall how exactly it looked in the survey anymore, but it’s in the results at least.)

Uh, I’m sorry. I posted I didn’t want to discuss it and here I went. I need to try harder.

2 Likes

I’m honestly a bit torn on this. On the one hand, properly asking such questions is hard, and getting useful data out of it even more. On the other hand, not surveying that data may make us blind to potential issues here. A lack of diversity is generally a problem in tech communities, and this might look like we are turning a blind eye to the issue.

Even though this year’s survey results are muddied by only 85% participation, one thing that might be insightful is to see how this compares to previous years. Perhaps we can at least somewhat reasonably track trends over time.

Extrapolating from my general experience with tech communities, I don’t find this improbable. It also roughly matches up with my perception of the NixCon audience FWIW (although I generally wouldn’t assume it being representative for the community as a whole).

3 Likes

baldness would be very small, because you tend not to tear your hair out using nix (only a little) :wink:

I’m one of the people who got confused by the nix vs nixos questions. Only now did I realise what was being asked (I think): do you use NixOS and/or do you use nix on non-NixOS systems. My mind went straight to “well, of course I use nix, I use NixOS, why is this being asked twice?”. If others were as confused or interpreted it like me, then the responses might be quite skewed.

Regarding the results, I find it curious that nix/NixOS is kind of “marketed” towards DevOps specialists, but less than a 5th of users identify as such. As somebody interested in such things, it is indeed unclear to me how to use nix/NixOS for deploying things for example.

It has been easier to present nix as something for developers to use, but the mix of nix-shell, nix develop, devenv, nigh completely undocumented yet widely used flakes just confuses them. I’ve stuck to “just use nix-shell if you’re making a package to be distributed and devenv if you want a developer environment”. Getting out of flake limbo / flimbo would be very beneficial as it seems devenv is built on top of it.

I also agree with the conclusion of the marketing team to scrap the “gender” question. @srid brings up a good point: why leave out the other group categories but keep gender? IMO, it’s irrelevant. If the nix community is welcoming and doesn’t treat others based on their appearances (which is difficult online unless you state it), it should be fine.

Thank you to the marketing team for the survey. Hopefully I’ll be able to participate next year :ping_pong:

5 Likes

Whereas I think this attitude is silly. Why check your oil if you aren’t doing a full 18-point maintenance checkup?

  • You suspect you’re running low on oil
  • Oil is the easiest thing to check yourself
  • Other reasons maybe, who cares, checking your oil can be important regardless of whether you’re also checking your transmission fluid and brake wear

The survey team should ask the questions to which they want to know the answers, based on the things they would recommend doing differently based on different responses. If one year that includes a gender question but not a race question, fine. If another year that includes a race question but not a gender question, also fine.

p.s. Thanks survey team, good survey good results. Keep doing what you’re doing.