Nix Community Survey 2023

Announcements
1

The Nix Marketing Team is pleased to announce the 2nd edition of the official Nix Community Survey! :tada: Please take 5-10 minutes to complete it.

https://survey.nixos.org/2023

Our community is growing very rapidly every month. This survey aims to gain insights into the the community members and what they care about the most when it comes to the Nix ecosystem. The information gathered from this survey will be used to improve Nix, Nixpkgs, and NixOS to better meet the needs of the community.

The responses are anonymized, we will NOT collect any personally identifying information (name, address, IP, …).

The questions in the survey cover:
• Your background (e.g. What region of the world are you from? What’s your current occupation?)
• The projects you use in the ecosystem and how often you use them (e.g. Are you using NixOS? How often do you use Nix?)
• How you use the projects (e.g. Do you use Nix in production servers?)
• What you like and dislike about the projects (e.g. What do you think is already great about Nix? What should contributors focus on improving?)

Last year, the Nix Community Survey 2022 received more than 2,000 responses! :exploding_head: We hope to make this year a success as well!

The Nix Community Survey is, in fact, a huge help for all maintainer and contributor teams. For instance, the Nix Documentation Team used it to plan and prioritize their work. Of course, the Nix Marketing Team also used this data to better understand the key strengths of Nix that can be promoted.

As usual, we are looking for feedback on the survey itself to understand how we can do this better in the future, don’t hesitate to send us a message.

The survey will be available until July 10th, we are eagerly waiting for your thoughts!

Thank you all for your contributions to making Nix better,

– The Nix Marketing Team

(Shout-out to @fricklerhandwerk for helping with limesurvey and improving the survey last minute)

51 Likes
2
3

Thank you for your work on this. Looking forward to the results.

4 Likes
4

Maybe the wording of the «feature» question should be different. Saying as someone whose top-3 critical features are really misaligned with the categories provided…

(Also, an ambiguity: features important to have in some form or features where improvement could bring benefits?)

1 Like
5

Thanks for the work put into the survey!

Still some suggestions for the future:

  • I think “Development Environments (nix-shell, nix shell)” should have been “Development Environments (nix-shell, nix develop)” and “nix develop” should be dropped from the “Building and packaging software”.
  • Some languages are missing, and as you clearly can’t serve everyones needs, perhaps add a free form field where poeple can add their languages if missing above? (in my case Erlang and Elixir are missing which are the most important actually) (similar for other “fixed” selection/ranking questions)
  • I wish there was a “magic wand” question in the “nix” section, not only in the NixOS section
6 Likes
6

Channels question: where is «none»? Hard to say which of the unstable channels best approximates «mainline checkout»

1 Like
7

Which changes to Nix would you most like to see implemented?

Ordering for this one is hard. ooi, what exactly is the difference between:

Granular incremental builds

Dynamic build graphcs (RFC 92, recursive Nix, …)

I read RFC 92 (first time I’ve heard about this, and it’s cool, goodbye bazel!), if I understand the first item correctly RFC 92 is how you would implement that?

2 Likes
8

“Which programming languages do you use in combination with Nix?” is missing Nim. We have had nimPackages for a few releases now.

1 Like
9

I appreciate that the survey was not too long, not too short! One thing that felt odd is that the “In what environments do you use NixOS?” question is missing an “Personal computer” answer. I answered “Development machines” but I do all my stuff on NixOS not just development. Or are you meant to leave it blank if you are, say, not a developer?

6 Likes
10

As others already said, many thanks indeed for the work put into the survey.

However, please be aware that not only is it arguably hard to order choices, but transforming the “preference lists” of individuals into a “preference list” of society, is fundamentally flawed in both theory and practice.

For a future survey I would suggest to let us grade each choice, as asked in most surveys. In the results, each choice would then get a profile of grades, and all choices be ranked using the Majority Judgment (ie. comparing using the median-grade, and breaking tie by removing (one-by-one) any grades equal to the shared median-grade from each tied choice’s). A method mathematically and experimentally proven to be “superior to any known method of voting and to any known method of judging competitions, in theory and in practice”, as their authors put it.

4 Likes
11

I would say not stating what is importance in the context of the question is a bigger issue.

It’s only best if the underlying preferences are actually in the «how many stars out of 10» format. Oh, they also assume comparable scales used by all the participants. Whether the preferences are closer to «order without values» or «numerical rating for each independently»… this is a separate (neurological, psychological, etc.) mess.

If preferences are ordinal (order of options without confidently held numeric values), then something Condorcet-based (like Schulze) is just … more applicable, and MJ failing majority criterion is — under this assumption! — unfortunate.

I suspect that the summarisation of the answers will be rough and numerical (not pick-one-first) enough that all the voting-theory things just don’t matter (but ambiguity in the question still do)

12

Hi everyone, thanks a lot for your on-point feedback. As @kranzes hinted at, this was done much more in a rush than I am comfortable with, and naturally there are mistakes and omissions. I’ll fix up as much as possible on the fly. Unfortunately it’s not possible to add questions or conditions once the survey is open, so we won’t be able to an input field for specific values of “other” options (in particular, for the ranking questions) in this survey any more.

Fixed.

Added Erlang, Elixir, Nim and Other as options.

While I agree this is great to vent some steam, free text answers are mostly useless for the purposes of this survey, because someone will have to process them and try to figure out patterns, adding work and bias. Order-of-magnitude numbers are good enough here. Also, the group of power users is small enough to discuss outlier positions directly or in person.

Added “none” option.

The “development machine” option is supposed to mean “computers you touch”. Aligned the answer options with the equivalent Nix question.

I’m aware of this, but here it really doesn’t matter: it’s not a vote. The purpose is to inform resource allocation decisions, and the ranking is there to nudge people to be specific. It’s easy to find everything important, and that doesn’t help with making decisions.

The questions clearly say “important for you”, so it’s about user preference. It admittedly does not capture what exactly you care about regarding these features, such as stability, performance, etc. but all this is only very approximate anyway.

7 Likes
13

It’s only best if the underlying preferences are actually in the «how many stars out of 10» format.
Oh, they also assume comparable scales used by all the participants.

Indeed, the Majority Judgment ballot aims at promoting interpersonal comparisons of grades, ie. an absolute scale using a common language, enabling people to judge in the absolute. Hence grading with numbers or stars is discouraged by the MJ in favor of using words (see section “9.3 Not faithful: dissimilar understandings of grades”). The signification of each grade can be explicited in a legend if need be, besides, the more those grades are used, the more their meaning is known and shared.

If preferences are ordinal (order of options without confidently held numeric values), then something Condorcet-based (like Schulze) is just … more applicable, and MJ failing majority criterion is — under this assumption! — unfortunate.

Those grades must be carefully chosen to induce people to express themselves in the absolute, that’s the crux, that’s one of the features of the Majority Judgment enabling it to escape Arrow’s impossibility theorem (voting theory’s most famous and most important result on how one alternative is to be selected among several) ie. to satisfy those 4 criterions when there’s at least 3 alternatives:

  1. neutrality: 1 person = 1 voice;
  2. transitivity: always being able to rank alternatives;
  3. consistency: being independent of irrelevant alternatives;
  4. unanimity (aka. Pareto optimality): ranking first any alternative considered better by all.

Condorcet-based methods all fail to escape Arrow’s impossibility theorem.

Even without ranking alternatives, having a profile of (carefully chosen) grades for an alternative already speaks volume: “Voting experiments have conformed with intuition: candidates may often be identified simply by the distributions of their grades (seeing how consensual, how polarized, how held in esteem or in contempt they may be)”.

Regarding the majority criterion, it is ill defined and largely incompatible with the Independence of Irrelevant Alternatives criterion.

14

Exactly, this is exactly the practically impossible part, which is why the theorems don’t fully apply to any practical situation.

Whether irrelevant alternatives or majority criterion is a bigger issue depends on how the options are created (it’s not an open registration, after all), how people’s opinions are structured, and whether you call the exact same outcome «just as planned» or «catastrophic failure» from the point of view of the long-term dynamics.

(Also, I have yet to see any voting system that satisfies transitivity as stated in your post).

There are cases where Majority judgement is the least problematic of the options, there cases where it is not, and the poll here is going to be used in an indirect enough way that all those comparisons don’t apply anyway

2 Likes
15

Regarding “How do you extend Nixpkgs”, one of the answers is “Flakes”, but Flakes don’t have a different mechanism for extending Nixpkgs. You still have to use overlays or patches.

9 Likes
16

Only if the extending you need to do changes parts of what’s already in nixpkgs. It’s perfectly valid to say that using a module from another repository, via flake (say, agenix), is “extending nixpkgs”, and doing so more easily / flexibly than other alternatives.

3 Likes
17

That is not flakes specific, as well, builtins.fetchurl is literally enough to do that.

1 Like
18

That’s just yet another mechanism to extend nixpkgs though, with its own benefits and downsides compared to flakes, niv & co.

Sure, all of them ultimately just throw something into a list of overlays or imports, but they are used quite differently and offer different kinds of support for the task.

19

Shouldn’t there be a banner on the NixOS website to inform readers of the survey? Otherwise, there is a risk of getting answers only from the most active community participants.

16 Likes
20

Agreed with the above, and also can I suggest to post this to link aggregators early on next time (HN, lobsters, lemmy?), to spread it more widely?

I’ve submitted it to both the above sites now, just in case.

6 Likes