Flag packages out of date

While maybe technically accurate in the sense of listed maintainers (I haven’t done a survey), I think this characterization might be a bit misleading. Many packages have listed maintainers, and I believe we’ve stopped accepting new packages that don’t have a listed maintainers. On top of that, there are many people with commit access that maintain lots of packages without being listed as a maintainer for it. During releases, we do Zero Hydra Failures where the community tries to fix as many packages as they can.

5 Likes

Your bot also does a great job at updating a lot of packages, but the reality is that we have a lot of “drive-by packagers” who see a package that is missing, make a PR to check it in, and then immediately abandon it. That is not really a profound problem, because none of these packages are critical.

3 Likes

Re: opening issues

I encourage to check out the following command in your local shell (provided gh cli installed):

gh issue list --label='9.needs: package (update)'

In a way we accepted for github to be our database backend.

I feel a bit offended by this statement. I’m trying over and over to move things into the right direction with my own time and I have a very poor success record.

As recently as today.

It’s not people. It’s organization.

2 Likes

You are referring to a completely different aspect. The present issue is about external tooling, not about altering the foundations of nixpkgs. The issue you’re trying to solve might be better tackled as an RFC. Trying to “shoot from the hip” with a PR for such things is generally bound to fail (see e.g. USE flags).

1 Like

I still feel offended. Not directly by you, but systemically by how things are.

From the viewpoint of the community as an (hypothetical) independent actor, there is no point in achieving “cheap” consensus about (a general idea of) resource scarcity and at the same time not achieving “effective” consensus about making practical and efficient use of the available ones.

It’s gross.

We can leave it here, since it’s off topic, as you correctly noted.

One of my stretch goals was to have you enter your maintainer name, and it would show you which of “your” packages are now failing, when they began failing, etc. From a UX standpoint, I’m still not certain how this should work, because of how many branches a user may care about (e.g. master, staging, staging-next, release-XXX). Also, people may only be a maintainer on a given branch…

Maybe I could have some repology / github integration which would also determine which are out-of-date as well.

Would be nice to bring more value in declaring yourself as a “maintainer” in nixpkgs.

5 Likes

Thanks for your replies!

  • @blaggacao Thanks for pointing me towards the github client, it seems a rather useful tool!
  • @jtojnar The repology notification tool is definitely a must have for maintainers! I will set this up for the one package I am maintaining at the moment :), there will hopefully be more.
  • And yes, it would also be great to have an “update request review” tool.

Nevertheless, I think the points are all made from the maintainers point of view. For users, maybe a clever GitHub Update Request Issue template would solve the additional boilerplate when reporting outdated packages. In the end, the act of “flagging” a package out of date feels more natural to me though.

1 Like

Personally, I am using nixos-search a lot. The best for me would be a button there (or on an equivalent other site if that matters).

EDIT: Flagging packages on nixos-search would even allow for branch-specific flags (well probably not branch-specific but channel specific).

For packages that have a standard expression format, and the build hasn’t fundamentally changed, they could just update the package doing:

git checkout -b bump-mypkg
nix-shell -p nix-update --run 'nix-update <pkg> --commit'
nix-build -A <pkg> # ensure it builds
git push <fork> bump-mypkg

and then create a PR with the bump. It is more involved, but less involved than finding where the nix expression lives, finding the latest version, update version, calculate new sha, building it, making a commit, then pushing to a fork.

For patch bumps, the above code is probably sufficient. For larger changes, there will some nix knowledge needed.

6 Likes

I agree this would be very useful. I think it would also be useful if you could add packages to this list of “your” packages manually: this way you could track packages that you care about (and might want to help get unstuck) even if you’re not ready to declare yourself maintainer.

(there’s some overlap with https://git.sr.ht/~raboof/nixpkgs-contributor-dashboard but I’m not actively working on that right now. Added a link to the README)

We could encourage to fork and open issues on that fork for that purpose.
That would fit nicely with gh issue status -R blaggacao/nixpkgs.

I don’t think I understand what you’re trying to propose…

Oh, ok, so here is another take from a different angle. Hope it makes sense.

Instead of inaugurating a sattelite database for the purpose of managing personal todo lists, I propose to stick with github, which the community by large has already adopted (for the worse or for the better).

This, as opposed to a sattelite data store, can efficiently leverage existing third-party tooling (gh) and intergrate well with existing gh enabled cli (or web ui based) workflows.

I’m aware that this perpetuates lock-in with github. Though I think consolidating workflow requirements (even around that tool), ultimately contributes to lower the switching costs.

A little oblique, but I’ve mentioned before that letting (encouraging/nudging?) people to indicate that they use or depend on packages might enable some interesting ~distributed levers.

  • I think it would need to focus on carrots at first. An example would be notifications when a package you depend on:
    • has been failing on hydra for [reasonable timeframe]
    • is updated (maybe some subset of updates?)
    • is marked/unmarked broken
    • becomes unmaintained or has a maintainer change
  • If the carrots do provide value and people bother signing up, it could build up a well of knowledge/experience that can help test changes, identify/develop tests that reflect real-world usage, locate new maintainers, etc.

→ I might assume that you refer to network effects?
(and occasionally/unaskedly would introduce that concept to the unbeknownst)

I do mean something different, though I didn’t intend to get deep enough into what is ultimately an aside to clarify how.

I'll tuck it into a details section to minimize disruption

There’s probably network-effecty value bound up in something like this, but it doesn’t strike me as the right focus or emphasis?

I focused the list of carrots on fairly straightforward individual benefits that might incentivize people to add themselves to a list without much concern for how or whether their total benefit scales with the length of the list.

My focus is on whether the project can add value above and beyond the time invested in creating and maintaining the system by building additional tools on top of it. (I mean something beyond just sending everyone an email and hoping a crowdsourced solution falls out.)

Some maintenance tasks are harder than necessary because the people performing them lack knowledge/experience that a package’s users have, with no great way to close the gap.

I called these “~distributed levers” because it’s probably a number of small components that, as a whole, make these lifts easier:

  • enable maintainers to indicate acute needs (ex: does this change work)
  • enable us to indicate systemic needs (ex: accumulate reports about whether something builds on a platform that isn’t marked supported)
  • nudge package users about this need at some sensible place in close proximity to their use of the package (i.e., before/after a build)
  • enable package users to collect and return the requested information in a quick low-friction manner that hopefully won’t derail their work with a big context switch
  • do something context-appropriate with whatever is collected
    • validate and propagate change test/feedback to the PR
    • validate a test case, generate a report, and write a patch/commit/PR for it
    • notify the maintainer that it’s available

I suspect experience/familiarity with a package would drive most of the value and that the value added wouldn’t continue scaling much with the number of participants once there are enough to get good, timely answers (and might roll off if low-familiarity users are providing incorrect or low-quality feedback).

Ah thanks! That totally clicked in my brain. Systemically, the issue at hand (in my words) is how to “shorten” or “close” that particular feedback loop.

We have two disparate knowledge domains, even three:

  • the ones that produce a particulat piece of software
  • the ones that package it for nix (our eternal bottleneck)
  • the ones that use it.

In my way of looking at this, producers and consumers are already sufficiently close in open source (thanks to github/etc). If we can eliminate the need for packagers to insert themselves into that (already established) feedback loop, we might have discovered a pathway to a solution.

I might been having a similar impetus while I was redacting this “sister” post:

is this still the best way to find outdated packages? edit: seems to be; sorry for noise.

I think so, yes. GitHub/Nixpkgs is the hub for all Nixpkgs issues including outdated packages. I have become used to it, but it may keep possible contributions of new users at bay…