Nix Community Survey 2023 Results

[Arsleust]

2023 Nix Survey Results

The Nix community is growing at an exciting pace, which makes it harder and more important to understand who comprises our community, what people use and care about, and the main pain points that frustrate users or impede progress and adoption. Therefore, the Nix Marketing Team conducts a yearly survey to better undestand these issues. Your feedback helps develop Nix, Nixpkgs, and NixOS to match your needs and come up with new ideas for better serving and growing the community.

tl;dr

• Compared to last year, we had more respondents, of which a good proportion were newcomers.
• The community may slowly trend towards more diversity in terms of technical background and occupation.
• The Nix ecosystem is still difficult to navigate and work with (developer experience, sparse or unclear documentation, lack of recipes)
• Nonetheless, most respondents consider that the benefits of using Nix/NixOS outweigh all the downsides.

We will try to make next year’s survey easier to complete and tailor it to better support the other community teams. Communication should also be improved to increase outreach and get more people to respond.

Total Participation, Response Rate

We received 1708 full responses, an increase of 15% compared to last year.

Taking into account the fact that most questions are conditionally prompted depending on a respondent previous answers, on average questions were answered 67% of the time. Without much surprise, the simplest questions (single choice questions) had the most engagement from respondents, but the order of the questions seem to matter as well (R²=0.30).

Warning

For many questions, respondents could select multiple choices. As such, in this analysis, percentages for such questions are percentages of “Yes” answers for the choice itself, represented in green. This explains why percentages for a seemingly unique question will not add up to 100%.

Single choice question are represented in blue, and the sum of all percentages is 100%.

The Shape of the Nix Community

Respondents were either from Europe (50%) or North America (23%) for the most part.

Given that many respondents did not state their gender (14%), there is a lot of uncertainty on the exact results. That being said, it is clear that most respondents were men (>80%). We may consider discarding this question, as did Stack Overflow and other surveys.

GNU/Linux is used by a large part of the community (80%), but other operating systems like Windows (27%) or macOS (26%) are also quite widespread. Surprisingly, about 50 respondents were Windows-only users.

Most people in the community are Backend Developers (31%), which is less than last year (was 40%, meaning a drop by 25%).

Overall, the “shape” of our community is largely the same as last year. However, we do see a reduced concentration of the previously most common profiles, which have a lower percentage than last year. One could say that the distribution is “flattening” a bit, which means that the Nix user base seems to be a bit more diverse. It will be interesting in coming years to see if this is an actual trend or not.

Nix at Work

The majority of respondents who use Nix do so on a daily basis (77%). As mentioned last year, this aligns with the expectation that those most interested in providing feedback are frequent users. Keep this in mind when reviewing the data and takeaways.

Most respondents have been using Nix for less than a year (38%), which hints at an increase of new users. Unfortunately we didn’t receive as many answers from users who have been using Nix for 1-3 years as we would have expected. Indeed, last year’s respondents who were, at the time, users with less than a year of experience should now have been part of the “1-3 years” group, making it grow accordingly. This would mean that many respondents with now 1-3 years of experience did not respond to the survey this year.

57% of respondents said they use Nix for work. At the same time, respondents said that they use Nix for at least one of their personal projects (88%). This could be interpreted that Nix users are very enthusiastic and want to use Nix, but either they deem it not ready for industrial use or Nix is not part of the tech stack at their workplace.

Key Themes

Last year, we had noticed 5 key themes:

• The “superpower”: Nix/NixOS is declarative and reproducible
• Nix/NixOS has a steep learning curve
• Nix/NixOS needs better documentation
• Nix is hard to debug
• It is hard to contribute back to the Nix ecosystem

This year is not too different: most Nix users love it for what it does well. Nix is used because it allows declarative configuration of reproducible systems and builds. Once you fix something, it stays fixed!

However, even the most enthusiastic Nix users agree on the fact that it has a very steep learning curve at every step. It is hard to get into Nix as a user, as a developer, and as a maintainer.

For each participant profile, people largely agree that the Nix documentation is one of the biggest issue. Sometimes because it is lacking, sometimes because it is unclear or not up to date. More specifically for beginners, it lacks comprehensive, accessible examples and recipes.

The barriers for wider adoption are that it is too difficult to make things work for particular use cases, or that documentation is not good enough.

Next Steps

Our goal is to help all of us understand what the community needs and wants. This information will benefit all parts of the Nix ecosystem, helping teams plan and prioritise efforts to meet the community’s needs.

We’ll be doing the Nix Community Survey again next year, and we’re working on making it even better. We gathered a lot of feedback from the last question of the survey, and plan to act on it.

I would like to thank everyone who helped make it happen yet again, and hope to see you all again next year!

Results

Below are all the quantitative results.

Method

The Nix Marketing Team conducted an online survey of 47 questions, of which 13 were single choice questions, 17 multiple choices, 4 ranking and 13 open-ended.

The survey was open for a one month period from June 11th until July 10th. We announced the survey by way of postings to Nix Discourse, Matrix, LinkedIn and Twitter.

The survey was split into four subsections:

• Background Information
• Nix User Questions
• NixOS User Questions
• Misc. Questions

Once the survey window was closed we split the analysis into two: multiple choice questions and open-ended questions. Multiple choice was analyzed numerically by compiling the responses by count and percentages, and comparing to previous year’s results. Open-ended questions were analyzed by manually reviewing the responses. The survey analysis was conducted throughout the month of August.

Final words

Huge thanks to all the people who contributed to the survey: @kranzes @ron @fricklerhandwerk @garbas @Arsleust

If you’d like to help organize next year’s survey please contact the marketing team on Matrix: #marketing:nixos.org

See you next year!

[Atemu]

A bit of unqualified feedback on the question design I’ve noticed while reading/answering:

• Perhaps add a Nix on NixOS option to the platforms question to avoid confusion. I imagine a significant number of people might not properly read the question and miss the part excepting NixOS. If they were instead presented with “Nix on NixOS” and “Nix on other Linux distros”, there would very little room for error.
• Use-cases: What use-case does using nix-shell/nix run for trying software without permanent installation correspond to? It’s not really “development” eventhough that’s where it says nix-shell but the tool isn’t listed under “Install software as a user” which it (IMO) actually fits in. Perhaps list nix-shell -p/nix run under “Install software as a user” aswell.
• Ask NixOS questions first to avoid confusing the user (ideally hidden if they don’t claim to use Nix on NixOS). I’ve been confused by two instances of the survey and had to revise answers due to this. Perhaps start with a question that is unquestionably about NixOS rather than Nix first (i.e. deployment tool) to prime the user.
• It might be a good idea to list a few very popular DIY DEs such as i3, sway or hyprland. When the most popular category with >30% is “Other”, that’s kinda weird.

[Arsleust]

We’ll definitely improve the situation with the Nix/NixOS sections, given some answers it must have been confusing to some.

Usually when an answer has too many responses which are “other” and it was possible to comment, we’ll see what were the most frequent responses and add them. Year after year, we should converge towards good lists of choices.

Thanks for the feedback, I really appreciate this

[vcunat]

Gender: Female < Other is intriguing to me, including the numbers. But I rather don’t want to discuss it, beyond just pointing it out. As noted, the root is probably hidden in the 14% who did not state their gender in a supposedly-anonymous survey (lack of trust, lack of perceived relevance?)

[piegames]

@Arsleust

Given that many respondents did not state their gender (14%), there is a lot of uncertainty on the exact results. That being said, it is clear that most respondents were men (>80%). We may consider discarding this question, as did Stack Overflow and other surveys.

I’m honestly a bit torn on this. On the one hand, properly asking such questions is hard, and getting useful data out of it even more. On the other hand, not surveying that data may make us blind to potential issues here. A lack of diversity is generally a problem in tech communities, and this might look like we are turning a blind eye to the issue.

Even though this year’s survey results are muddied by only 85% participation, one thing that might be insightful is to see how this compares to previous years. Perhaps we can at least somewhat reasonably track trends over time.

@vcunat

Gender: Female < Other is intriguing to me, including the numbers.

Extrapolating from my general experience with tech communities, I don’t find this improbable. It also roughly matches up with my perception of the NixCon audience FWIW (although I generally wouldn’t assume it being representative for the community as a whole).

[nixinator]

baldness would be very small, because you tend not to tear your hair out using nix (only a little)

[UefiPls]

I’m one of the people who got confused by the nix vs nixos questions. Only now did I realise what was being asked (I think): do you use NixOS and/or do you use nix on non-NixOS systems. My mind went straight to “well, of course I use nix, I use NixOS, why is this being asked twice?”. If others were as confused or interpreted it like me, then the responses might be quite skewed.

Regarding the results, I find it curious that nix/NixOS is kind of “marketed” towards DevOps specialists, but less than a 5th of users identify as such. As somebody interested in such things, it is indeed unclear to me how to use nix/NixOS for deploying things for example.

It has been easier to present nix as something for developers to use, but the mix of nix-shell, nix develop, devenv, nigh completely undocumented yet widely used flakes just confuses them. I’ve stuck to “just use nix-shell if you’re making a package to be distributed and devenv if you want a developer environment”. Getting out of flake limbo / flimbo would be very beneficial as it seems devenv is built on top of it.

I also agree with the conclusion of the marketing team to scrap the “gender” question. @srid brings up a good point: why leave out the other group categories but keep gender? IMO, it’s irrelevant. If the nix community is welcoming and doesn’t treat others based on their appearances (which is difficult online unless you state it), it should be fine.

Thank you to the marketing team for the survey. Hopefully I’ll be able to participate next year

[rhendric]

@UefiPls

why leave out the other group categories but keep gender?

Whereas I think this attitude is silly. Why check your oil if you aren’t doing a full 18-point maintenance checkup?

• You suspect you’re running low on oil
• Oil is the easiest thing to check yourself
• Other reasons maybe, who cares, checking your oil can be important regardless of whether you’re also checking your transmission fluid and brake wear

The survey team should ask the questions to which they want to know the answers, based on the things they would recommend doing differently based on different responses. If one year that includes a gender question but not a race question, fine. If another year that includes a race question but not a gender question, also fine.

p.s. Thanks survey team, good survey good results. Keep doing what you’re doing.

[Sandro]

@Arsleust

It is hard to contribute back to the Nix ecosystem

A few months back I wanted to get a patch into Arch and failed because I wasn’t a trusted member. My work has a really hard time to upstream Openstack fixes even though multiple people work on it a considerable amount of time. If you want to contribute to the kernel you better have a mail address at the right provider.

When people tell me they found a typo in search.nixos.org, I can tell them click on source, find the place again, change it, create a PR, someone probably merges it in a few days.

[waffle8946]

@Sandro

@Arsleust

It is hard to contribute back to the Nix ecosystem

When people tell me they found a typo in search.nixos.org , I can tell them click on source, find the place again, change it, create a PR, someone probably merges it in a few days.

Missing the point, as it’s not a process issue, but a skill issue. Nix is an extremely simple language, but packaging things with nix quickly becomes a nightmare, especially with docs that are lacking. Many (new) users still believe packaging requires a PR to nixpkgs, or that flakes will solve their packaging woes, then they run into a brick wall.

And as new users were overrepresented here, naturally “it is hard to contribute” is a common complaint.

[Sandro]

@waffle8946

but packaging things with nix quickly becomes a nightmare

Yes and no. Nix is just brutally in uncovering wrong usages in build systems like autoconf or cmake, assumptions about the distro you are using or shortcuts like downloading random things in the build process. If you have a package which closely follows the recommendation and documentation of your build system then it probably just works by putting the software upstream says it requires into the inputs.
The nightmare comes when people write custom build scripts and don’t use the standard things which they then only ever tested on a specific distro eg. Ubuntu. Or to make it worse: it only builds in their Docker container.
Those things are then first noticed when using nix as it doesn’t allow to use many dirty shortcuts commonly used when just hacking something together on another distro. This is a great and terrible feature at the same time and makes things for beginners certainly not easier but it also prevents breaking the declarative nature of nix and confusing it.

Also to bring in my favorite distro Debian: You can have long docs and the process can still be a complete nightmare, making it even for general Linux experts a pretty hard task to build a debian package the debian way.
And Debian has pretty terrible docs, too but people just accepted it and the entire internet is filled with answers to their questions from the last 20 years. For example I wanted to install postgresql without creating a database and also needed to look into their packaging how to disable it. The answer was to run a perl script to modify a config file from and in the postgresql-common package.

Also it sounds like you are a newer user to nix. Why don’t you contribute to the docs and fill in the one or two holes you noticed while getting into it? Even adding one sentence somewhere can make the difference . People that are kinda fresh are great at spotting when something important is missing. Unlike people like me that are pretty into and who will probably never notice when some important reference or explanation is missing especially in the beginner parts of the docs.

@waffle8946

Many (new) users still believe packaging requires a PR to nixpkgs, or that flakes will solve their packaging woes, then they run into a brick wall.

This means we should probably check if the beginner docs mention how to create custom packages or overlays.

[Sandro]

@cafkafk

I’d take a step back and say “what are factors that prevent people in general”.

NixOS is a pretty technical distro and a large part of the community is technically minded people. Almost all really technically minded groups I can think of in my surroundings, be it work, associations or collectives, have similar problems and to me it feels like a bigger societal issue that society fails to give all people the same opportunities and chances to get into technical things which propagated into our community. That’s why it is so hard to fix this because the problem itself is gigantic and pretty much everywhere.
But it feels like in the last years diversity grows which also propagates into nix. There are just so many things to do everywhere and you don’t even know where to start.

Also in a more specific note: I feel like beginners especially not male identifying ones often compare themselves to the person that know nix/nixos best around them which is obviously a bad comparison which is not encouraging. Also they often don’t find the courage to ask people around them how to tackle, solve and break down a scary looking big task into smaller steps which are easier to solve and what’s the thought process behind it. I think it is always important, if time allows it, to give people the steps so that they can learn how to solve things on their own and are not reliant on others to help them craft a line of nix code.

[waffle8946]

@Sandro

Nix is just brutally in uncovering wrong usages in build systems like autoconf or cmake, assumptions about the distro you are using or shortcuts like downloading random things in the build process.

I don’t disagree, nix is more of a magnifying glass on the problem, not the problem itself. My only point was - because nix doesn’t let you take certain shortcuts, new users find it hard to contribute.

[NotAShelf]

I’ve been around Nix/NixOS long enough that I no longer consider myself a beginner. Yet I still find the documentation to be severely lacking regardless of difficulty of the topic the documentation in question is meant to address

On another hand, Ppeople who primarily contribute to documentation are those who know exactly what they are doing - which usually means assumptions are made when determining what is common knowledge and what is not. I think this is natural, and more or less unavoidable since the status quo seems to have been so for a long time now. The only solution I can think of would be a complete overhaul and unification of documentation. By which I mean make EVERYTHING accessible in one place and eredicate nixos.wiki.

Edit: I think the debian example is a bit irrelevant. It sounds more like an excuse why we can’t improve than what I think it’s meant to be.

[cafkafk]

@NotAShelf

The only solution I can think of would be a complete overhaul and unification of documentation. By which I mean make EVERYTHING accessible in one place and eredicate nixos.wiki.

I mean, nixos.wiki has some flaws for sure, but it is still useful, at least to me. Also, I have to admit that I do think the documentation is okay if you view it as just a reference and not a learning guide. What I feel really is lacking is motivated examples and demonstrations. Stuff like NixOS in production is a great step in the right direction.

I think a reason NixOS struggles with this is that how people actually structure their system configuration can be very different, and that makes it harder to share general patterns. For Arch Linux, you mostly just have to document what series of commands to run, whereas with Nix, the users need to think more about the actual architecture of their configurations.

I think the debian example is a bit irrelevant. It sounds more like an excuse why we can’t improve than what I think it’s meant to be.

I mean, I think it’s a good example, and it demonstrates that it’s not necessarily just because packaging with Nix is inherently harder. So if that’s the case, why are people frustrated with the experience? Part of it is probably documentation, but what else?

[waffle8946]

@cafkafk

So if that’s the case, why are people frustrated with the experience?

Because as mentioned above, nix doesn’t allow the usual shortcuts. I can cheat and download things in the build() step of a makepkg script on Arch Linux, while that doesn’t fly here in the sandbox.

Also, new users think the wiki is official, and the wiki is therefore the biggest source of nix and nixos disinformation. Then they get frustrated when they think they’ve followed “official docs” and get issues. And they therefore find it hard to see an “entry point” into using nix.

[michaelCTS]

@Sandro

When people tell me they found a typo in search.nixos.org , I can tell them click on source, find the place again, change it, create a PR, someone probably merges it in a few days.

I’m not sure if that’s the norm. I have made a few PRs in various nix projects and the majority are stuck in review or haven’t been reviewed at all despite asking for reviews (this isn’t my only account for contributions).

The presence of the threads for people to request reviews or share PRs that have been reviewed but haven’t been merged, is IMO a clear indicator of either lacking or incomplete process. It’s not necessarily saying nix is worse than projects of its size (I wouldn’t know, this is the first big project I’m able to contribute to), but as usual, everything has room for improvement.

One big thing I noticed is that there doesn’t seem to be a way to move from a forum post proposing improvements to actually implementing them. It’s not clear who owns what, who can be talked to to get a ball rolling, what the process is for improvements, etc. Key/buzzwords for me are org chart and official wiki.

[vcunat]

I believe this is a topic for a different thread, but I’m very doubtful how process can solve the fact that you can’t really force volunteers to work on packages that they’re not interested in (by which I mean: review, merge and take responsibility, etc.)

Of course I understand that it’s painful, but it’s not so one-sided IMHO. I can imagine that processes could improve the problem through improving things like (1) the set of people with push access, (2) more granular access, (3) better ways of matching PRs with reviewers, …