Incident: some custom notifications for nixpkgs have been lost

If you have nixpkgs-(committers/maintainers) access and use custom notifications on nixpkgs, those have unfortunately been lost and you have to re-configure them back.

For transparency purposes, here is what happened:

Yesterday I got a report from a user that nixpkgs-committers couldn’t close issues and needed triage access. This seemed like a thing nixpkgs-committers should be able to do.

This morning I changed the permissions quickly, in order to unblock the user.

Then two hours later, I panicked a bit when I saw users reporting they lost write access, ending up switching the nixpkgs-maintainers as well.

So things are back to as they were before:

  • nixpkgs-committers: write
  • nixpkgs-maintainers: triage

Unfortunately, and something I didn’t know, is that GitHub reset custom notifications for those users.

23 Likes

Does that mean I have to re-subscribe to PR’s and issues I was watching?

From what I gathered, this only affects users that had set custom notifications here:

image

6 Likes

I am fairly certain that I hadn’t explicitly set Custom notifications for Nixpkgs, since I didn’t know this option exists. I still got set to All Activity though.

6 Likes

Since today noon (CET) I receive a lot of email notifications from nixpkgs that I’m not tagged in. I’m guessing this is the root cause, but I still need to figure out how to unsubscribe. :smile:

Ah. It was suddenly set to “All activity”. Thx for the screenshot.
So same as khumba. (Should have read the entire thread first)

Google’s cache of the NixOS/nixpkgs repository shows 181 watchers on 22 Mar 2022 08:09:34 GMT.

Right now it’s at 659 watchers. Something resulted in 500+ people likely being subscribed to every single GitHub event on the nixpkgs repository.

Going by my own inbox unread count before I clicked unwatch, that’s at least 85.000 notification emails…

4 Likes

i have created an github ticket asking them if it is possible to rollback the notification settings

11 Likes

Thanks for your transparency @zimbatm! Accidents happen to the best of us!

9 Likes

Did your ticket get resolved @Artturin? Because my custom settings stayed as they were.

3 Likes

Thanks @Artturin !

I must admit to not reviewing my settings carefully. For anyone who hasn’t found it yet, it is currently the first setting at github account settings for notifications.

image

The picture shows the relevant setting: “Automatically watch repositories”.

3 Likes