How many people are paid to work on Nix/Nixpkgs?

Development
6

What would really help if someone coded a github actions that would automate the rfc process.

A 2 minutes search yielded: GitHub - rust-lang/rfcbot-rs: Coordinates asynchronous decision making on Rust repositories. Status of tracked issues and PRs can be viewed at https://rfcbot.rs.

Who is going to tackle this wish? (I’m not capable in doing it.)

3 Likes
7

I think the current model is more bottom-up. Individuals or companies see value in some stuff (monetary or other), so they improve it. Even changes primarily intended for their own usage get often contributed (easier long-term maintenance).

Originally the foundation was just owning and buying most(?) of the build farm, not much else IIRC. I’m not aware of any changes there, but I haven’t been following closely in the past couple years.

I actually like the bottom-up approach, including RFCs for bigger changes. I believe a single leader can’t in principle have enough information to just make decisions that are good for “majority” of the large community, and they also need to somehow persuade people to implement those decisions. I’m not saying it can’t work the other way around – top-down decisions from some leader (or tiny group) with a firm vision and a very “loyal” contributor community – but I think by that the “scope” of the project gets smaller.

Maybe this example is a little unfair, but I believe Guix is determined to make it hard to use unfree packages and that drives many potential users away. (I mean, even if they strongly prefer libre stuff, people get often “externally forced” into using a couple proprietary packages.)

12 Likes
8

they also need to somehow persuade people to implement those decisions.

This is so much more important than making decisions quickly, provided they are made. And it takes time for people to “sleep over” something a couple of days / weeks.

I think of this in terms of Work the Least for It. Because it minimizes friction.

The key point will be to resolve divergent opinions quickly and promote mind share and consensus. This translates into momentum. And momentum into action.

Maybe an aid would be to stratify the discussion so that priorities don’t get out of sight. What do I mean? → Strategic Disscussions, Meta Discussions, Governance Discussions, Developmemt Discussions, etc.

Currently it’s still a bit blurry and discussions mostly either atart or terminate around technical aspects. This is obviously how it needs to be, mostly.

But raising awareness for the other levels of discussions might result in more opoortunities for Work the Least for It.

9

I like this approach @blaggacao. Let’s take as an example the mentioned rust bootstrap PR. After some technical reviews, pretty much after this comment, all of the participants mentioned some of the imperfections in the proposed changes, and they debated how best these will be handled. They all said it would be a shame if the PR would bit rot, and it sort of did but thankfully the author is still willing to put work into it.

I see this PR as a typical example where a small detail was mishandled in the PR by some opinions, and eventually the PR author feels neglected because their work is not merged because of such small imperfection. In that Rust’s PR case, I’m sure the author of it would have performed the proper changes to it if only someone was “brave” enough to say “Do this and you are good to go”. See also discussion here.

Maybe having 1 and only leader that will decide on every debate we have in our PRs won’t be beneficial, as different domains require different areas of expertise. But I do think e.g the rust ecosystem, and other ecosystems needs a single “leader” (or call it however you want) that will cut such debates quicker.

I agree, that’s why I tend to think now after hearing you all, that we need final decision makers (final authorities) only for every domain of Nixpkgs (or for every NixOS team).

When I imagine a leader of such a team, I imagine someone with self confidence that is not afraid to disappoint some people for the sake of what they believe in. It should be someone with a vision and their goals should be public. I also imagine this person taking hard decision faster then in an RFC, because they are targeted.

It is possible they will not take the best decision in every debate, and it should be possible to veto their decision if you think it’s absolutely awful. But perhaps it’s better to take a bad decision vs not taking a decision at all?

3 Likes
10

I strongly disagree with this. In the past two decades or so, I have contributed bits to other distributions. Many distributions are organized like this, having ‘leaders’ of subareas. Some subareas become small personal kingdoms and nothing ever gets done at all because their leaders reject or stall everything. Either because they like to exercise their powers or because the see every change as a personal attack on how they did things.

NixOS moves very fast compared to other distributions. Not only in terms of package versioning, but also in terms of how large infrastructure changes are adopted. And I believe that this is exactly because the Nix ecosystem has a relatively flat hierarchy and everything that is considered an improvement is generally accepted (if there is someone who has the time to review it).

I agree that we have a problem in that some PRs get stuck, and we should address that. But having gatekeepers will only make things worse.

Also, I believe that in this discussion a lot of emphasis is put on PRs that are stuck because nobody seems to be willing to make a decision. That happens, but the vast majority of our PRs are stuck because we do not have enough people to review PRs.

9 Likes
11

Also, I believe that in this discussion a lot of emphasis is put on PRs
that are stuck because nobody seems to be willing to make a decision.

I think this is a consequence of 2 things:

  1. We generally have nice people in our community that don’t want to piss somebody else off

  2. We don’t have formal, explicit documentation of workflows (the why we do things in a certain way) which means no way to escalate

I have merged stuff that had quite some disagreements with a “unless somebody comes up with a compelling argument against this 1 week from now, I’ll go ahead and merge this”. At the same time, I have also accommodated changes that I disagreed with simply because the other person cared more about this specific issue than I did.

It would make things easier if we could simply say “please make these changes because this document tells us to do so”. It would also be great to be able to ping somebody in case of conflict (conflict is a little to harsh for this admittedly), which would allow us to do “I want this and XXX wants that. @somebody, please pick one”.

We have old PRs from both old and new contributors that have gotten stuck and it is a terrible signal to send when we want more hands on deck (hey, I’m guilty of this too).

14 Likes
12

Yes, and it’s even worse than that. GuixSD makes it hard to use OpenZFS even though OpenZFS is free (and indeed copylefted). There are arguably good reasons for that, but the fact remains that it’s not only unfree software that’s excluded.

1 Like
13

I’m of a few minds about this and don’t have conclusions or a point… sketching the territory for my own benefit and sharing just-because.

I see a few things that feel like axioms here:

  1. There’s a limit to how much work-someone-doesn’t-want-to-do that they’ll do. This limit will ebb and flow with health and other commitments.
  2. There’s a finite amount of work we can all get done, regardless.
  3. Project-management/org/workflow decisions can:
    • increase or decrease the total amount of work that must be done
    • affect the amount and type of work getting allocated to individuals
    • “change the weather” in a way that shifts our individual limits in one direction or another

Presumably, the most-ideal outcomes for any “Project-management/org/workflow decisions” are a pileline that:

  • Doesn’t leak work that would otherwise benefit the community.
  • Doesn’t leak people who will otherwise benefit the community.
  • Does leak/shed/drop work that won’t benefit the community (low quality, high maintenance, misfeature etc.), but strikes some balance between limiting the total amount of energy these eat up and “leaking people”.
  • Resists clogging. (All causes! Collective indecision, local tyrants, trouble finding knowledgeable people to review or escalate to, people needing to take some time off or step away entirely with no one to stand in).
  • Minimizes duplicate effort. (I mean less “let’s have 3 reviewers” and more “this issue got read 100 times by people trying to run issue triage and never became actionable” or “this issue got bikeshedded for 2 years and never came to a clear conclusion”.)
  • Doesn’t naively allocate much work-someone-doesn’t-want-to-do to people who won’t/don’t want to do it.
  • Doesn’t default towards overloading and running off the most-productive maintainers with energy-sapping meta-work (unless we’re pretty sure the meta work is the bigger lever, and the maintainer wants to do it).
  • Defaults towards protecting uninterrupted blocks of time instead of becoming an interruption firehose.
12 Likes
14

Re: both:

A clear set of max 10 behavioral rules which are - first priority - effective against clogging!

@abathur your list actually sounds like excellent abstract requirements for a draft RFC on the topic.

I’d add: label-triage for humans based on empowered, educated and understandable reasoning is key vs the cryptographic approach :cry: .

I mean you just start to ignore labels when most of the time it looks like this with no actionable value for a human being.

image
image1639×490 71.6 KB

1 Like
15

I find these labels very useful when reviewing:

  • The Linux/Darwin counts:
    • Tell you how many other packages depend on this change and thus how large the impact would be if the PR breaks something. Of course, leaf packages can be important to large groups of users too, but the counts are somewhat of a reasonable proxy of what the impact of merging PR is going to be.
    • Give a ball-park estimation of how many packages your machine will be building when you want to nixpkgs-review that PR. If this is a extremely large number, you probably want to cherry-pick a set of dependent packages that are either important or representative.
    • Tell you whether the PR should target master or staging.
  • The by-package-maintainer label is useful, because these PRs are typically quicker reviews (unless they introduce a new package). A maintainer typically has a lot of domain-specific experience, is likely to have tried a new version somewhat exhaustively, and knows the quirks of the software.

I agree that there are other useful possible labels (waiting review, waiting for author response, waiting to be merged, etc.) But the existing labels are very useful, at least to me.

5 Likes
16

I suspect many straightforward approaches will place values on this list in tension/conflict, so much of its value (if it has any) may come from what it rules out. Process changes that reduce individual/global workload and spread it around better are hopefully welcome, but I also think there are some real risks, like:

  • expending effort just to relocate the clogs (i.e., from collective indecision over a PR to collective indecision over how long we should give a previously-stellar-but-now-burned-out code-owner to address rule on the PR before giving someone else their turf)
  • converting tasks people are doing with enthusiasm but not consistency/efficiency into a bureaucratic process that they avoid, or into a regular notification-nag that keeps them from focusing on other work
2 Likes
17

You are right, I didn’t mean to question the theoretic usefulness of the labels, that is their information value.

I meant to question the practical usefulness in social context, that is their informational value + the ability to interpret them.

This is why I labeled them provocatively as cryptographic: You need decoder knowledge (like many do). :wink:

It could also be:

  • impact/S, impact/M, …
  • maintainer-advantage

You might notice the differenc in stance. It makes them automatically more human friendly.

I sometimes feel labels might have been abused as stateful data store for the ofborg because they seem to change in arbitrary non-intuitive fashions.

I also had a discussion with @timokau on this about marvin labels (“LastMile” / “LastMeter”). That discussion supports where I’m coming from and where I’m aiming at.

But it’s a secondary (even terciary) hurdle, if existing labels only would be used more actively.

1 Like
18

I sound’s like you stress pareto-efficiency here. I think you are right in doing so!

Without some sort of elevated powers, non-pareto-efficient agreements aren’t possible.

I’d argue, RFCs are the elevated power and the place to “piss somebody off” without acrually pissing them off.

19

+1 to seeing if there could be improvements to the review process.

I sort of think we should just auto approve r-ryantm PR’s that are older than a week old. They’re basically the best tested PRs in existence + I think they’re cognitively expensive to review compared to their risk.

20

I like the idea. @ryantm, why not? If those are mostly good why not inverse the odds? (and be a bit pushy on the maintainers, who’d else review them on unstable)

21

Btw if anyone can lift me past the hurdle for making nix-review more useful…
https://github.com/Mic92/nixpkgs-review/pull/117#issuecomment-665389853

It’s kind of blocking my attempt to make the PR templates more user friendly and add more automatism - for the occasional leaf packages only.

22

I look at every r-ryantm PR and read all the comments on them, and I think probably about 5 to 15% of them are bad in some way. The issues include: dependency changes, runtime breakages, important reverse dependencies failing, Darwin/ARM failures, needing to be updated as part of a group, already part of another more comprehensive PR, etc.

I think it really does take human knowledge and intuition to decide if they are okay to merge.

I could see this changing if we had more package level tests. For example, I’d expect it to be safe to auto-merge one where the package and all its reverse dependencies had tests and the tests passed.

Ryan

14 Likes
23

Yeah. I’ve wished we had a way to nudge people using under-tested packages for test cases (and for them to submit, and for us to collect…)

2 Likes
24

Personally, for all the packages I maintain, I’ve added sanity checks so I’d be able to stay confident that @r-ryantm’s PRs haven’t broken anything (e.g this).

I once sketched in my head a policy we could adopt, where packages could hold a meta field that will enable PRs automatically merge by ofborg once they pass it’s tests along with @r-ryantm’s reverse dependencies tests. Thus maintainers would hopefully be encouraged to add this meta field as long is there’s an appropriate check phase of some sort.

RFC 50 wanted to address that but it got stall because of disagreements upon the kind of policy the bot would implement.

Even in the current state, we are ranked as the top updated repo at repology. Thanks @ryantm :heart:.

4 Likes
25

I don’t know. This gives upstream maintainers more or less the power to directly push into nixpkgs by creating a tag. It only requires one malicious actor or compromised GitHub account to push a backdoor or trojan into nixpkgs.

When reviewing PRs I try to (where humanly possible) do a diff between upstream versions to quickly glance if there are no fishy things going on. I also run binaries in a very restrictive bubblewrap sandbox to check if they attempt require strange things (I also plan to automatically extract a list of accessed paths, attempted connections, etc.).

I think there should be a human in the loop, so that the chance to catch something malicious (or just a stupid error) is increased.

IMO this is one of the tasks of a Linux distribution, to ensure that the users get vetted, working, packages. If that is not doable anymore, then perhaps we are packaging more than we can manage.

I completely agree that it would be great if maintainers could merge PRs against their packages. I think this speeds up the process and will keep more maintainers around, since they are more empowered to actually maintain their packages.

6 Likes