21.05 Retrospective


To have a discussion around the stable release, and how to improve the process. This will be very similar in intent to the last retrospective.


We will hold a 1 hour meeting to facilitate questions and have a discussion around what didn’t work well and how to improve. The issues will likely be time-box’d (5mins initial discussion, with 2 minute extensions) to keep pace of discussion going and not end up just talking about one topic the entire time. (This is a rule-of-thumb, but the exact protocol is subject to change, I just found timebox’ing to be much more effective in real-life).

Topics should be categorized into 3 sections:

  • What went well?
  • What do you feel neutral about?
  • What did you feel negative about?

I will aggregate topics from this thread, (if i can find a decent tool) participants will be able to vote for 3-5 (depending on number of topics) topics that they would most like to discuss, and proceed with time-boxed discussion. If we don’t discuss a particular topic you were interested in, this doesn’t mean it wasn’t important, we just didn’t around to it in this meeting. If we find these meetings to be super-productive, then we may hold additional meetings.

The desired outcome is to have some “action items” which can improve the next NixOS release.

When / Where ?

The retrospective will be held 2021-06-06T15:00:00Z2021-06-06T16:00:00Z. We will use Matrix, I will give the specific room details before the meeting.

I’m willing to push back the time up to 4 hours, I would just like for US / EU individuals to have the opportunity to join. (Sorry Aussies and Kiwis).

This time, I will verify that my OBS has the web browser as an audio source :frowning:


Anyone who attempted to contribute, contributed, uses, or has valuable feedback on the 21.05 release.

Policy for this Discourse thread

Please feel free to add anything to the following topics in a post:

  • What went well?
  • What do you feel neutral about?
  • What did you feel negative about?

You can also start a discussion to other peoples posts, however, if a particular topic warrants it’s own thread, then it should probably be moved into it’s own thread. I would like to mainly keep this thread focused on discussion topics for the retrospective meeting.

cc marketing team @garbas @grahamc @ryantm (sorry if i forgot some people)


Forgot to kick off the discussion (this will later be used during the meeting, the discourse thread is largely for people to be come aware of the issues and start to formulate their own opinions):

What I liked:

  • RFC80 and RFC85 seemed to have reduced the risk going into the release signifcantly
  • Saw a lot more darwin build fixes for ZHF (thanks @stephank and others)
  • Native M1 Support
  • ZHF on master seemed to have worked really well (way less tedious than 20.09 and prior backporting)
  • Didn’t have to spend any time on plasma. (thanks @ttuegel)

What I feel neutral about:

  • No defined process for onboarding new platforms (e.g. aarch64-darwin) to hydra cc @grahamc @domenkozar
  • No defined support tier support for platforms, which further complicates the above issue. We have language to describe tier support from RFC46, but no current list of what platforms fall under which tier.

What I feel negative about:

  • Losing WorldOfPeace was big blow to the gnome team: @jtojnar had to pick up more slack. We should probably make a community effort to package and stabilize gnome for 21.11 and on.
  • x86_64-darwin hydra jobs were delaying the ability to do staging-next cycles in a timely fashion.
  • Could have had another week “post-branch-off” to further stabilize release notes, hydra builds (especially the stable staging-next), and do some more QA. (e.g. graphical iso was broken)
  • llvmPackages refactor (although well-intentioned) did cause 100+ downstream failures. I would like to add llvm to be added to release critical packages and for the definition of “breaking changes” to include restructuring outputs of the packages as well. Also, any changes made to llvmPackages is a darwin-stdenv-rebuild, which further complicated issues with x86_64-darwin stabilization.
  • I also didn’t make great use of release-engineers. I was also closing on a house, and didn’t have as much free-time so I prioritized reviewing PRs and solitary release work.

people are free to post here :). If you’re not able to make the meeting, you can still have the items discussed.

I found it difficult to get started because of my lack of experience with Hydra.


(from ZERO Hydra Failures 21.05 · Issue #122042 · NixOS/nixpkgs · GitHub):

How to help (textual)

The “textual” was confusing, as I wasn’t sure if it meant that there was going to be a video as well.

Select an evaluation of the trunk jobset

I didn’t know which evaluation I should choose. Should I always choose the last?

Find a failed job

Should I click on “Still failing jobs”? Or “Evaluation errors”? (In retrospect, it is obvious now, but it wasn’t 100% back then)

you can use the filter field to scope packages to your platform

It took me a while to figure out that there is no drop-down menu, and that I have to filter with a text field. How to do this was not obvious to me though.

The true thing though is that I was quite busy, and could have figured these things out on my own (or asked). That’s why I didn’t bring them up during ZHF. But I thought it might be useful feedback nevertheless.

Thanks for all the great job you’ve done so far streamlining and clarifying the process!


Retrospective link: https://matrix.to/#/!fnslRFjkYdpEWafotY:nixos.org?via=nixos.org&via=matrix.org&via=monoid.al :slight_smile:

Action items from retrospective:

  • Mac drive or fundrasing to acquire more hardware @grahamc
  • Discourse thread on tier support @jonringer @grahamc
    • Revisiting RFC46’s appendix based real-world events
  • release-wiki PR to add llvmPackages to release-critical @jonringer
  • Discourse thread on hydra platform onboarding @grahamc
  • Discourse thread on stabilization teams @jonringer
  • Call for release manager 21.11 @jonringer
    • Expand on benefits of being a RM
    • Include changes from RFC85
  • Hydra support for ZHF
    • Better UI for contributors (finding relevant failures)
    • Search by maintainer?

Review board: 21.05 Retrospective - Google Sheets


I wonder if someone could reach out to of the mac hosting providers to sponsor us for macOS when we need to scale up, similarly to how Equinix Metal supports us for Linux?

MacStadium offers free dedicated Mac hosting for opensource developers. Case Study: Homebrew