Hello everyone, you’re friendly neighborhood release manager here.
Just wanted to bring some clarity around what the release schedule timeline will before the 21.05 release. This is heavily based upon RFC 85 which aims at decreasing the amount of potential breakages going into a release by limiting changes to
staging to just minor bumps and bugfixes. Large breaking changes right before release has caused many issues in the past, and they should be avoided.
For simplicity, I broke out the events into two tables, one for the release centric events, and one for “restricting” breaking changes done to the
||Zero Hydra Failures (ZHF) Begins, targets master
||Branch-off, move to “backport” ZHF workflow targeting release-21.05
||21.05 Release, ZHF Ends
||Staging / Staging-next Event
||Breaking changes to release critical packages are not allowed to be merged
||Breaking changes to any package which needs to target
staging is disallowed
staging-next PR, begin ZHF next day
||Merge first stable
||Start second stable
staging up to any breaking changes, these changes will not land in the release
||Merge second stabilizing
||Branch-off, release development will follow
||21.05 Release, ZHF Ends
stable staging-next PR just means a
staging-next PR whose changes will be included in the release.
ZHF: Zero Hydra Failures, a period in time in which we try to fix as many failing builds and tests as reported by Hydra.
Does this mean we don’t have to backport PRs for the first two weeks of ZHF?
- Yes, part of the motivation was to make ZHF less tedious for both contributors, and reviewers. The first two weeks usually takes care of most of the “low hanging fruit” for fixing builds and tests, only a minority of fixes will need to be backported after the branch-off.
Will my contributions to
master be affected?
- No, as long as you are “green to green” (package and downstream packages still work) or “red to green” (fixed a package or downstream issue), you will not see any impedance. This is largely the case for most contributions targeting
Why limit breaking changes to only
- Staging remains to be the “Achilles’ Heel”(weakspot) of nixpkgs. Changes which target
staging generally affect several hundred to thousands of packages. Trying to capture these effects is difficult for any distribution; and generally handled by seldomly updating these types of packages (e.g. Debian 10 vs Debian 11). For the stable release, I wanted to mitigate the number of “new failures” which may had already been fixed in previous ZHF PRs.
- PRs targeting master are essentially “small enough” in scope that the changes can be mostly captured, thus these PRs do not need to be restricted and likely trying to restrict them in some manner will likely cause more issues than it will solve.
Can I add a new version of a package that would normally warrant a staging PR?
- Yes. For example, if gcc11 were to be released, you will be able to add
gcc11Stdenv into nixpkgs; however, you would not be able to bump the default
gcc, as this would cause a mass rebuild and likely introduce many regressions. You could also view the PR to add
gcc11 as a PR which would target
master, therefore doesn’t warrant any breaking changes restrictions.
Branch-off will occur when
x86_64-darwin builds get below <6k, which is usually a large number of leaf packages. (haskell alone is ~6k).
Other interesting point. In the week that we had aarch64-darwin builds (M1 builds), we now surpass homebrew in up-to-date package count, and have roughly 3x the package count . Obviously quantity isn’t everything, but I think it’s a testament to nixpkgs that a new architecture can be onboarded so quickly, and make great use of a lot of prior packaging work.
Want to thank @thefloweringash for enabling the aarch64-darwin bootstrap and additional stabilization. Thanks to @grahamc for enabling this scenario on the infra side :).
And thanks to Rob for getting us 6 M1 mac minis months ago
release-21.05 has been created, and branch-off has occurred. ZHF will now transition to a backports workflow. More details with be in the ZHF issue.
One week til release (assuming nothing catastrophic happens).
Release happening today, as originally planned?
Unfortunately, there’s still about 12k queued builds for x86_64-darwin. Hydra - Jobset nixpkgs:staging-next-21.05. It will probably be finalized on Monday.
cc @garbas @domenkozar @vcunat
Speaking of the announcement… I guess aarch64-darwin won’t be “officially announced” yet and remain in some kind of experimental mode? (I think ATM it’s built in no jobset but
staging-next and there’s no channel.)
M1 is supported in regards to: people don’t have to bootstrap their system, apple SDK 13 has been added, big sur is supported (IIRC). There is still on going discussions around what level of support we will have going forward; currently there is only a
staging-next jobset building their builds, which means that updates to master or
release-21.05 won’t have new builds until the
staging-next branches get merged.
cc @thefloweringash for specific details on M1
cc @grahamc on nixpkgs support
Seeing as the darwin package set has made little progress, and that darwin is a lesser supported platform; I will be cutting the release regardless on Monday.
The majority of stabilization that we will see in this release has already been applied; and the linux package sets have been completed since Thursday.
If anyone would like to make the case on blocking the NixOS release on darwin build progress, please do so here. I already mentioned boosting darwin resources for the next ZHF period, but that’s not feasible to do for this release.
I think delaying the release 3 days was a warranted compromise, and any further delay would be detrimental to the NixOS release. It will be the first release since 17.03 which will have released in the month it was intended.
I am in favor of releasing as is, let’s do a retrospective after the release to figure out how to do better next time.
21.05 has been released! The tag has been created. And the channels now have 21.05.
I was having some issues creating the nixos-homepage PR, so there will be an official announcement tomorrow when @garbas is available.
Thank you very much indeed for all your work!
Thank you Jon and everyone involved in this release!
Congrats on a huge, on-time release!
When updating using nixos-21.05 release nixos-21.05.3980.f0869b1a2c0,
21.05beta555.d25ea6a0d2a: is there something missing in the release or the channel is not updated yet?
Noticed as well, went with small instead 21.05-small
The stable tag has been created, and the channels have been marked with
stableBranch. Just need the jobset to complete again. For example, nixos-small has already completed: nixos-21.05-small release nixos-21.05.608.4eb41db7d8f
Thanks for all the hard work on this release. I just want to say that having watched, and participated (lightly) in a couple different releases, this one seemed a lot smoother overall and I think that’s largely due to the changes in the release process. I think it’s been a big improvement, including unstable being a lot more stable through the process.
I know a lot of people are responsible for this so thanks to all of you.
21.05 has now been officially released: 21.05 Has been Released!