Maybe the wording of the «feature» question should be different. Saying as someone whose top-3 critical features are really misaligned with the categories provided…
(Also, an ambiguity: features important to have in some form or features where improvement could bring benefits?)
Thanks for the work put into the survey!
Still some suggestions for the future:
Channels question: where is «none»? Hard to say which of the unstable channels best approximates «mainline checkout»
Which changes to Nix would you most like to see implemented?
Ordering for this one is hard. ooi, what exactly is the difference between:
Granular incremental builds
Dynamic build graphcs (RFC 92, recursive Nix, …)
I read RFC 92 (first time I’ve heard about this, and it’s cool, goodbye bazel!), if I understand the first item correctly RFC 92 is how you would implement that?
“Which programming languages do you use in combination with Nix?” is missing Nim. We have had nimPackages for a few releases now.
I appreciate that the survey was not too long, not too short! One thing that felt odd is that the “In what environments do you use NixOS?” question is missing an “Personal computer” answer. I answered “Development machines” but I do all my stuff on NixOS not just development. Or are you meant to leave it blank if you are, say, not a developer?
As others already said, many thanks indeed for the work put into the survey.
However, please be aware that not only is it arguably hard to order choices, but transforming the “preference lists” of individuals into a “preference list” of society, is fundamentally flawed in both theory and practice.
For a future survey I would suggest to let us grade each choice, as asked in most surveys. In the results, each choice would then get a profile of grades, and all choices be ranked using the Majority Judgment (ie. comparing using the median-grade, and breaking tie by removing (one-by-one) any grades equal to the shared median-grade from each tied choice’s). A method mathematically and experimentally proven to be “superior to any known method of voting and to any known method of judging competitions, in theory and in practice”, as their authors put it.
I would say not stating what is importance in the context of the question is a bigger issue.
It’s only best if the underlying preferences are actually in the «how many stars out of 10» format. Oh, they also assume comparable scales used by all the participants. Whether the preferences are closer to «order without values» or «numerical rating for each independently»… this is a separate (neurological, psychological, etc.) mess.
If preferences are ordinal (order of options without confidently held numeric values), then something Condorcet-based (like Schulze) is just … more applicable, and MJ failing majority criterion is — under this assumption! — unfortunate.
I suspect that the summarisation of the answers will be rough and numerical (not pick-one-first) enough that all the voting-theory things just don’t matter (but ambiguity in the question still do)
Hi everyone, thanks a lot for your on-point feedback. As @kranzes hinted at, this was done much more in a rush than I am comfortable with, and naturally there are mistakes and omissions. I’ll fix up as much as possible on the fly. Unfortunately it’s not possible to add questions or conditions once the survey is open, so we won’t be able to an input field for specific values of “other” options (in particular, for the ranking questions) in this survey any more.
Fixed.
Added Erlang, Elixir, Nim and Other as options.
While I agree this is great to vent some steam, free text answers are mostly useless for the purposes of this survey, because someone will have to process them and try to figure out patterns, adding work and bias. Order-of-magnitude numbers are good enough here. Also, the group of power users is small enough to discuss outlier positions directly or in person.
Added “none” option.
The “development machine” option is supposed to mean “computers you touch”. Aligned the answer options with the equivalent Nix question.
I’m aware of this, but here it really doesn’t matter: it’s not a vote. The purpose is to inform resource allocation decisions, and the ranking is there to nudge people to be specific. It’s easy to find everything important, and that doesn’t help with making decisions.
The questions clearly say “important for you”, so it’s about user preference. It admittedly does not capture what exactly you care about regarding these features, such as stability, performance, etc. but all this is only very approximate anyway.
It’s only best if the underlying preferences are actually in the «how many stars out of 10» format.
Oh, they also assume comparable scales used by all the participants.
Indeed, the Majority Judgment ballot aims at promoting interpersonal comparisons of grades, ie. an absolute scale using a common language, enabling people to judge in the absolute. Hence grading with numbers or stars is discouraged by the MJ in favor of using words (see section “9.3 Not faithful: dissimilar understandings of grades”). The signification of each grade can be explicited in a legend if need be, besides, the more those grades are used, the more their meaning is known and shared.
If preferences are ordinal (order of options without confidently held numeric values), then something Condorcet-based (like Schulze) is just … more applicable, and MJ failing majority criterion is — under this assumption! — unfortunate.
Those grades must be carefully chosen to induce people to express themselves in the absolute, that’s the crux, that’s one of the features of the Majority Judgment enabling it to escape Arrow’s impossibility theorem (voting theory’s most famous and most important result on how one alternative is to be selected among several) ie. to satisfy those 4 criterions when there’s at least 3 alternatives:
Condorcet-based methods all fail to escape Arrow’s impossibility theorem.
Even without ranking alternatives, having a profile of (carefully chosen) grades for an alternative already speaks volume: “Voting experiments have conformed with intuition: candidates may often be identified simply by the distributions of their grades (seeing how consensual, how polarized, how held in esteem or in contempt they may be)”.
Regarding the majority criterion, it is ill defined and largely incompatible with the Independence of Irrelevant Alternatives criterion.
Exactly, this is exactly the practically impossible part, which is why the theorems don’t fully apply to any practical situation.
Whether irrelevant alternatives or majority criterion is a bigger issue depends on how the options are created (it’s not an open registration, after all), how people’s opinions are structured, and whether you call the exact same outcome «just as planned» or «catastrophic failure» from the point of view of the long-term dynamics.
(Also, I have yet to see any voting system that satisfies transitivity as stated in your post).
There are cases where Majority judgement is the least problematic of the options, there cases where it is not, and the poll here is going to be used in an indirect enough way that all those comparisons don’t apply anyway
Regarding “How do you extend Nixpkgs”, one of the answers is “Flakes”, but Flakes don’t have a different mechanism for extending Nixpkgs. You still have to use overlays or patches.
Only if the extending you need to do changes parts of what’s already in nixpkgs. It’s perfectly valid to say that using a module from another repository, via flake (say, agenix), is “extending nixpkgs”, and doing so more easily / flexibly than other alternatives.
That is not flakes specific, as well, builtins.fetchurl is literally enough to do that.
That’s just yet another mechanism to extend nixpkgs though, with its own benefits and downsides compared to flakes, niv & co.
Sure, all of them ultimately just throw something into a list of overlays or imports, but they are used quite differently and offer different kinds of support for the task.
Shouldn’t there be a banner on the NixOS website to inform readers of the survey? Otherwise, there is a risk of getting answers only from the most active community participants.
Agreed with the above, and also can I suggest to post this to link aggregators early on next time (HN, lobsters, lemmy?), to spread it more widely?
I’ve submitted it to both the above sites now, just in case.
Were the results of this year’s survey ever shared?
Evaluation is not finished yet. The marketing team does all of this on their free time, so it takes a while.