@musicmatzementioned on Mastodon that we recently hit 4000 open nixpkgs issues, which made me curious about the trends in those numbers. I wrote a quick script to get them from the API and break them down per month, giving:
Issues
The below graph shows cumulative open issues (blue, positive) and closed issues (purple, negative) over time.
So last month was really good (a spike in opened issues, but an even larger spike in closed ones), but overall both seem to grow pretty linearly.
(columns are: opened in that month, closed in that month, difference, cumulative open, cumulative closed)
Actually I think PRâs are a more interesting metric. itâs kinda fine when obscure issues hang around for a long time, while a contribution going unmerged is more of a problem.
Since nixpkgs accepts so many PRâs I cut the graph in 2 so we can see trends more clearly, but keep in mind the scales are different:
I think the major thing that nixpkgs is missing is quick triaging. Thereâs a lot of PRs that could be merged if they followed CONTRIBUTING.md and the nixpkgs manual more closely. Itâs very time consuming to review ~20 PRs on small items, and then to have them and 30 other PRs get pushed onto my ânotificationsâ backlog.
I tried to start with some âeducationalâ videos on nixpkgs on the PR process:
Another issue that thereâs no consensus as to what is âgood enoughâ for nixpkgs. So Iâve also seen many times where an author gets pulled by 3 different reviewers.
Ultimately, we just need more people reviewing PRs (not necessarily committers, but it can be frustrating to have a PR polished and no committer to push the button). But the work is largely unrewarding (unless it affects you), and can be very time-intensive.
Another problem is that many maintainers seem to be inactive. Since they have added themselves to a package, they are probably interested in the software and possibly stakeholders, so they are the ideal reviewers. But my feeling is that the majority of PRs (I may be wrong here, it would be nice to get some statistics) are never reviewed by the maintainers that âownâ the package.
I often feel a bit reluctant to review updates to such packages, since I donât want to get in the maintainerâs way. I now usually look when they last committed and if itâs a longer time ago Iâll review the PR anyway.
I am not sure what the solution to this is, but it would be nice if we could convince maintainers to stay active over longer periods. It would be nice have an idea why they became inactive. Do they not use nixpkgs/NixOS anymore? Did they lose interest in the packages and forgot to remove themselves? Do they find getting their reviewed PRs merged to frustrating?
It would be great if it was possible to âsubscribeâ to a package more dynamically. Interest doesnât map 1:1 with the git history. Sometimes I package a thing because I want to go in a direction, then loose interest.
If there was a tool that would go over all the packages that I use and publish those somewhere, I would be more than fine with it. It could then be used for all sorts of things; package popularity, notify of security issues, and let me know when a package that I use has an issue.
I wonder if we could extend hydra to make this happen, you can register to follow failures of certain package names, and get emails if thereâs a failure on a jobset/branch of your choice.
The other issue/question becomes, how is this different than the current meta.maintainers, and why do we have two similar but different abstractions for package maintenance.
Once upon a time (until ~March 2018), this new feature you want was actually the definition of meta.maintainers. It has been disabled for reasons I am certainly not going to be the best at explaining but, I think, boil down to âtoo many false positives and potentially buggy implementation.â
One idea that has been spitballed is to make one RSS feed per package so that people would be able to subscribe to the ones of interest to them, an idea which would have less catastrophic failure modes than emails (I personally received 194 emails on 2018-03-11, the day itâs been disabled, and AFAIR it stopped at 194 only because someone manually killed postfix)⌠but it needs someone actually implementing it.
Itâs been brought up by multiple people that werenât aware that a package they maintained was broken, and would have liked to have been notified.
I think there is a need for maintainers to be aware of breakages. How itâs implemented I think is the hardest question. Iâm not a big fan of using email either, as my personal email is listed under my maintainer entry, and the first thing i would probably due is make a rule to put all the failed notifications in a directory Iâll rarely or never touch.
Edit:
However, if i was able to visit nixpkgs:trunk on hydra, and it told me which of the âstill failing packagesâ were mine, I think that would be useful.
How about a single GitHub issue for (package, current version, problem) tuple automatically created by builder?
Check if the issue exists, if open or closed (implying false alarm), do nothing.
If doesnât exist, create and add maintainers as notifyees.
Remove maintainers as watchers immediately.
3 is important because itâs too much noise (which you can expect to see on the issue from causual observers looking to get their problem solved).
The tuple could even be just (package, current version).
The goal is to make only so much noise that a maintainer notices. Given the deluge of emails, it might be better to not notify the maintainers than notify too much.
Iâm not the biggest fan of leveraging github issues. My github notifications are already increasing at a unmanageable rate being a python codeowner, anymore and Iâm likely to backburner even more PRs.
Not to mention that nixpkgs issues is already at 4000+ open issues
iâm a newcomer to nix, maintainer of three packages today, but for a long time maintainer of just one package. when some months ago i first received an email requesting a review to a pr updating that package, i was in doubt if i really should post a review or if that request was an automatic message not really addressed to me.
since iâm new on this community, with few contributions and no built trust, i was doubtful if my review would be either useful or expected. i had believed that only the reviews of committers or experienced users would be considered. just when i read some posts/documentations asking to even new users make reviews, i was sure to make one.
i cannot speak on behalf of other maintainers, but at least for me, if i had sooner received some message or read in someplace that my reviews would be useful even i being a newcomer, i would had started to post reviews to my packages earlier.
Maybe we could make this information more explicit the newcomers.
Thanks for the work. People raising the question of high count on different channels almost always met with âitâs because repo is too active, X amount of issues closed just last monthâ argument. Looks like the trend reflects the feeling.
At the time of writing around 12.5% of all issues were automatically generated by @ckauhaus, and about 7.5% of all PRâs were generated by r-ryantm. Another 6% of issues are packaging requests.
I wonder if it makes sense to give maintainer reviews more prominence by adding a âaccepted-by-maintainerâ label once one of the maintainers has submitted an approving review. First of all, such a label would signal that reviews by maintainers are appreciated. Secondly, such a label may result in quicker merges, since they indicate that a PRs is already validated by a domain expert of the package.
Not sure if this is directly relevant but there is also a factor of âcleanliness begets cleanlinessâ I think?
I wonder if it would help if, say, only actual (or potentual) bugs existed on GitHub and questions and feature /packaging requests were suggested to be posted here or programmatically migrated here,?
Does GitHub have a way for admins to set a âdefault viewâ for issues and PRs? It might also help to only have bugs show in the default view?
Or perhaps close issue tracker to everyone but actual commiters (+ experts ) and redirect people to discourse for everything⌠Verified problems can be graduated to actual bugs on GitHub by people with permissions. (GitHub doesnât have this feature though⌠, Or a way to limit issue opening at all, I will research if amy other bug-tracker has restrictions, although here the relative lack of usage of other platforms might act in favor of the concept )
