Nixpkgs Architecture Team conclusion and prospective

Around when RFC 140 was accepted, participation in the Nixpkgs Architecture Team (NAT) fell to a lower level, where the “Team” format was not working out anymore.

As the team lead I’ve thus decided to officially dissolve the team.

This sound more dramatic than it really is though, because not much will actually change. In this post I’ll go over how it has come to this, what to learn from it, and future plans to look out for.

History

Out of frustration with various architectural problems in Nixpkgs I created the NAT in July 2022 and subsequently got some members to join me. We started having weekly meetings, all recorded and with meeting notes. Asynchronous discussions were being held on Matrix, with a separate GitHub organisation for more persistency.

After an initial period of various brainstorming discussions, we started focusing on fixing a single problem, which is now known as pkgs/by-name, established by RFC 140. While very slow and tedious both before, during and after the RFC, I’m happy to say that this has been a definite success. It’s not done yet, I’m continuing to work on it, but expect it to be done Soon™. If you’re interested about more technical details, I gave a presentation about this last NixCon.

Unfortunately some time along the way the team meetings started falling apart. Around the time of the RFC, we were struggling to find much to discuss in the weekly meetings, they often just turned into me monologuing what I’ve been working on. Because of that, the meetings were at first reduced to just 30 minutes, then made bi-weekly, and then just once every 4 weeks. The intention was to schedule longer in-depth discussion meetings when necessary, though that never really happened. And with meetings 4 weeks apart, it’s easy to forget about it, and a single absence now meant a two month period without synchronisation.

I’ve also grown increasingly grumpy as a team lead, since I’ve been doing the vast majority of the work:

  • I administered the team effectively by myself: Leading every meeting, taking almost all meeting notes, pinging people in advance to join, keeping team information up-to-date, etc. Attempts to delegate such tasks have failed for various reasons.
  • I wrote RFC 140 mostly by myself, with only @roberth helping out a bunch (appreciated!). While the RFC was discussed in meetings and on Matrix, actually putting these discussions in concrete writing was left mostly to me.
  • I’m implementing the RFC entirely by myself, without much feedback in PRs. This is at least partially due to it being a lot of Rust and not Nix code. Only since recently I’ve gotten some very thorough reviews by @PhilipTaron (appreciated!).

Of course I can’t expect anybody else to actually do anything since I’m not paying anybody. I’m in the lucky position to be sponsored for this (thanks Tweag and Antithesis!), but this is rare. In the end, this just didn’t feel like a team at all anymore and made it hard to stay motivated.

Challenges

Lack of authority

A core problem from the start has always been the lack of authority: The team used the community-based RFC process to enact changes and had no concrete ownership of Nixpkgs’ architecture, despite the name. Considering that there are over 200 Nixpkgs committers without a dedicated owner, this seemed like the best way to make substantial progress. And while this works, it’s very slow and tedious. While it’s very beneficial to write down decisions and involve the community in that way, at some point somebody should just be empowered to make decisions.

Lack of resources

As in almost every corner of Nix, there’s a shortage of resources. This is especially problematic for Nixpkgs’ architecture, which is notoriously hard to change due to the scale of Nixpkgs, with over a thousand PRs being merged every week. The vast majority of which add and maintain packages, making heavy use of the architecture. This effort is largely problem-oriented, where people do the work to solve their own problem, but don’t have the time and knowledge to do more. To make any significant changes, somebody with these resources needs to take the initiative and push it through. While admirable, we cannot expect people to undertake such efforts for free and should instead hope for this to be sponsored.

Plan

The following will continue to exist:

In the following weeks I will do these changes to dissolve the team:

Prospects

To work towards improving the authority situation, I wrote an open Nix organisation draft proposal, which is being experimented with by @zimbatm and me. This wouldn’t directly fix such problems, but rather create a place to discuss and propose changes to it. Feel free to discuss this idea by opening issues/PRs or in the Nix Platform Governance Matrix room.

Not having the burden of trying to manage this team structure anymore allows me to focus more on actual work. I fully intend to both finish the implementation of RFC 140 (see this milestone for the status), and continue to work on Nixpkgs’ architecture after that.

Furthermore I’ll start holding a weekly personal office hour, where anybody can come talk to me about anything relating to Nixpkgs’ architecture, but preferably with a focus on what I’m working on at the moment. I haven’t decided a time yet, but if you’re interested you can enter your availability here:

47 Likes

Thank you a much for your patience and diligence @Infinisil! Apart from solving a very old, very annoying problem, and demonstrating how this can be done in the most considerate, non-distruptive, sustainable way, your work clearly shows what it takes to steer a ship like Nixpkgs – and that we need more captains and officers of your kind.

And thanks to everyone who participated on the way, even if "grumpy Silvan” ended up doing most of the heavy lifting. Those discussions, however tedious they seem, are vital to make sure the right things get done right.

16 Likes

I’ve now taken all steps to conclude the team:

  • The NAT is now removed from the website
  • The GitHub issues have been migrated to a new architecture label under Nixpkgs
  • The private Matrix channel is now publicly archived and deleted
    • Note that this was only possible because no private information was shared in the room, this doesn’t have to become a precedent!
  • The GitHub organisation is now archived with a link to this thread
  • The YouTube channel was updated to link to this thread
  • The meetings are removed from the NixOS calendar

Furthermore I scheduled my personal weekly Nixpkgs architecture office hour (ping @PhilipTaron, @Lyndeno), see Community Calendar - #120 by Infinisil

7 Likes

Hi Silvan, thank you very much for all work you did and keep doing! I fully understand your decision and I think it makes a sense in current sutuation.

1 Like

Hi! So is it too late to ask if you want help :smile: ?

Either way just wanted to say I saw your not-all-packages-anymore.nix talk and enjoyed it!

@willbush Thanks! I’d appreciate help, here’s some pointers:

  • Join the Nixpkgs Architecture Matrix channel or my new weekly office hour to synchronise with me and others for all of the below

  • If you know Rust and are interested in helping out with the pkgs/by-name effort specifically, check out the RFC 140 milestone for things to do. While I’m planning to work on most of them myself, if I could offload some of that, it would help a lot! Otherwise, reviews are very appreciated too.

    Furthermore you can also just look at the nixpkgs-check-by-name codebase and make suggestions for improvements.

    This all will be easier to help out with once nixpkgs-check-by-name is in its own repository.

  • As mentioned above, there’s the new architecture label in Nixpkgs. These are open architectural problems left to fix, each of which requiring lots of resources.

    • If you see an appropriate issue not labeled yet (hard to keep track of them!), or if you create a new issue for something that’s not tracked yet, let me or another Nixpkgs committer know, so it can be labeled
    • If something interests you or you have ideas, engage with comments
    • The most impactful, but also most difficult way to help, is to pick up one of these issues and work on a design document/proposal to address it. This might be classified more as research rather than work, and you need to understand the topic very well to make good progress with it.
2 Likes

@Infinisil Thanks for the response! I’d like to help with the pkgs/by-name Rust project. I do have some experience with Rust. Mostly toy projects, but I did make a slack bot for work and CLI tool that sends out messages over RabbitMQ that we use all the time. However, rust isn’t our main language unfortunately. I was able to get nixpkgs-check-by-name up and running / tests passing thanks to the declarative nix environment and direnv.

The project is very clean and well tested! Nice work! I guess I’ll start looking at nixpkgs-check-by-name fails for `poetry2nix.mkPoetryApplication` packages created in `pkgs/by-name/` · Issue #261052 · NixOS/nixpkgs · GitHub if that’s ok? I’m actually surprised how few outstanding issues remain.

Awesome!

Yeah the issue there is something that would be nice to have fixed at some point.

I also just opened `pkgs/by-name` errors are inconsistent · Issue #296335 · NixOS/nixpkgs · GitHub for something that’s more tractable :slight_smile:

I’m fairly new to Nix* but I’m slowly starting to grasp how much work the chunk of the problem you addressed is. I have to say that I have had multiple “oh really?”-moments to last month or so. Great job!

For the Youtube link: it appears to not contain an actual link but just plain text. I don’t know if this is intentional though.

1 Like

@pie Not exactly sure what you mean by “oh really?”-moments tbh!

For the Youtube link: it appears to not contain an actual link but just plain text. I don’t know if this is intentional though.

The one in the initial post (https://www.youtube.com/@nixpkgs-architecture) works for me, which link doesn’t work for you?

Not exactly sure what you mean by “oh really?”-moments tbh!

For example: I thought oh that would be a few pull requests - a little bit more if it is more split up - to implement. But RFC 140 Milestone · GitHub says otherwise. I’m still young and inexperienced in software engineering though so that might just be me :smirk: .

The one in the initial post (https://www.youtube.com/@nixpkgs-architecture ) works for me, which link doesn’t work for you?

I’m sorry… I mean the one on Youtube, linking to this thread. But now it strikes me that that may be a limitation of Youtube itself.

1 Like

Ahh yeah good point! While links within the Description don’t appear to work, I was able to create a link explicitly now (this might be a very recent feature) :slight_smile:

2 Likes