Also, maybe not only the release date should be shifted, but branch-off should be 40–60 days before the hoped release date, not 20–40? Buys a few rebuild iterations when something tries to refuse to play nicely.
Ideally, of course, big changes would land a bit earlier in advance at unstable (as in: with all Hydra-built tests passing, not just merged), but maybe branch-off deadline will see less slip than «please leave some time for stabilisation before branch-off»
There’s a separate discussion around staging and staging-next. But having 500 largely untested staging commits land a few days or the day prior to the target branch-off did contribute to some of the delays.
Gnome 3.38 was merged into master before 20.09 was released, so I think that’s low risk for impacting a November release date. Plasma + kde + Qt, can also be stabilized before ZHF, one of the compounding issues with ZHF, is that you’re risk adverse to potential regressions at that point. But if we have a working plasma installation, then it’s easier to backport individual kde/qt fixes. Also, during ZHF, a lot of core contributors are “spread thinner” due to a lot of backports occurring in many different ecosystems.
Another way to word my proposal is, “The desktop experience in nixpkgs-unstable should be pretty good by the time November rolls around, so we branch-off, we can just focus on the general package regressions in master and not have to try to fix large package ecosytems while doing stabilization.”
It was not merged into master, it was merged into staging. And only approximately one day before the 20.09 release. As a GNOME maintainer, I am not really confident of including GNOME 3.38 before running it on production system at least a month.
Yes, they do, each cycle there are few issues that are discovered by Arch, Fedora or Ubuntu people and for which we do cherry-pick patches (less than ten, this cycle, IIRC). Some of them are included into .1 release two weeks after .0 but some of them don’t.
But majority of the issues, and the hardest ones to solve come from integration with NixOS. For example, we have decided to not include the new search in GTK file dialogue for now because waiting for that would delay the merge even further. See GNOME 3.38 · GitHub to get some idea what GNOME update entails.
Hopefully, we will solve these before the end of November but, even if we do yy.11 releases, the GNOME changes would have missed the branch-off point.
While not that gcc packages, GNOME developers do develop some more widely used packages like gtk-doc or Vala, which sometimes do break stuff. GNOME can depend on the latest version of those. And I would not be surprised if they even needed last version of systemd, PulseAudio or other freedesktop.org projects since their developer pools overlap, and GNOME developers often add new features to those projects, driven by the needs of GNOME desktop.
You bring up a lot of good points.
But I would just like to point out that our 5.18.5 plasma and 3.36 gnome which we released with 20.09 still have many rough edges. Even a week after releasing, I’m still largely doing, “damage control”.
The point I’m trying to make is, we would still be in much better of a state than we were with the early September branch off.
Although there still hasn’t been any 5.20 plasma development on master; if we “get things rolling” with a well-structured project/issue (similar to Convert remaining python2 applications over to python3 · Issue #101964 · NixOS/nixpkgs · GitHub), and maybe a special branch+ hydra job, then it “easier” to distribute the load with the community.
I’ve been looking into the release cycles of both Plasma and Gnome 40.0. Gnome follows a 6 month release schedule, however, plasma has a 4 month release cadence for their non-LTS releases. This means we will have a three month period between stable releases where our stable release will have an unsupported version of plasma. Personally, I don’t think this is much of an issue; unless there’s a compelling security reason, the plasma desktop should still be very usable. I would much rather have people saying, “why isn’t plasma at the latest”, than “why can’t I log out”.
Doesn’t plasma release an LTS version? why don’t we use that?
They do, but even with the 20.09 release, we had issues with systemd compatibility where these regressions did not occur on the later plasma releases as they were developed against more recent versions of systemd (logout, and certain functionality was missing or broken). Subtle incompatibility issues between these desktop managers and fundamental libraries will likely always occur, however, the greater the divergence, the more likely we will have to develop in-house solutions. Thus delaying releases more.
As I’m not a user of gnome or plasma, I would like some feedback from those users and maintainers about their thoughts. My main interest is being able to release in a timely manner, and the Desktop experience is usually people’s “first impression” of NixOS. With the YY.05 and YY.11, I think it’s very doable to have a “mostly polished” experience going into the release branch-off. Which should save everyone a lot of headache, pain, sweat, and tears.
TL;DR
Going with YY.05 and YY.11 release dates with non-LTS versions of plasma and gnome will have an awkward window for plasma where there will be a 3 month window where it will be unsupported. However, I don’t think this will be much of an issue as long as we can provide a very usable desktop experience.
Hi, user/admin here. As I administer some machines for very non-tech-savvy users (using plasma and xfce, not gnome), I would also opt for a stable plasma installation and would not mind if it is not the newest version. After a new NixOS release got published, I usually need some weeks to prepare/update individual configurations for my users. Beyond that, their systems run on auto-update. New features in new plasma versions are rarely missed; stability is more important.
This also holds for most other software: I don’t mind having older versions of libreoffice, latex, but also server software like sshd or httpd, although these probably need more attention with respect to security updates.
I hope this data point helps finding a solution. Thanks for all your efforts!
As someone coming from Arch, my ideal would be a very good, stable unstable 
. Basically inverting what I understand as the current focus, where unstable is less tested and the releases like 20.03, 20.09 are supposed to be more tested.
That said I’m probably in the minority there. More realistically as a Gnome user I’d prefer working slightly out of date Gnome versus up to date but broken in strange ways Gnome. Hope that helps.
As someone coming from Arch, my ideal would be a very good, stable unstable
I think in this discussion people like me who care nothing about releases just do not see any reason to participate! People who for whatever reason want to have releases with desktop support discuss how to do this better, I do not expect to either do anything for releases nor benefit from releases, so I just do not see anything useful I could say…
I will continue building my system from random master snapshots and fix what breaks for me…
That said I’m probably in the minority there. More realistically as a Gnome user I’d prefer working slightly out of date Gnome versus up to date but broken in strange ways Gnome. Hope that helps.
OK, Gnome is harder… StumpWM-related breakages I can fix literally on my own in a day…
If I was dictator, I would just post-pone our staging cycle (except for regression fixes) during ZHF and have ZHF target master. Branch-off from a really good point in master, do a final week of backporting and QA, then releasing.
Then do all the “risky” staging work after branch-off. Or at least until we have a practical staging-next stabilization which is treated more like a “release” with blocking jobs.
Yes! Shorter time between branch-off and release but with a freeze period for core changes before it.
Of course at that point those running master/unstable will be more affected.
Ideally master would be “well polished” at that point. Do you mind clarifying how unstable users would be more affected?
I also think that stability is more important, especially in nixos: if I really want one recent software, I can still install that software from unstable and keep my system on stable. Also, I’m curious to know: is there enough man power to deal with two releases per year? I’ve the feeling that it takes quite some energy to release and maintain a stable branch (needs backports, but the backports cannot usually be done without some changes…), and I don’t know if it is really important to have two releases per year. And if this energy could be used to maintain a more stable unstable branch, I would say it could be interesting.
The branches diverge over time, at a certain point, it becomes non-trivial to backport changes from master into the release branch. This is also why NixOS will probably never do a 4-5 yr LTS. That’s just too much upkeep, and probably making in-house patches for stuff to still build.
Yes, but I think we just have to do the practical thing about what we are supporting. Most distros don’t ship with 17 desktop managers, 34 window managers, and 60k+ packages. One of the reasons why I was proposing the delay to YY.05 and YY.11 release dates was to help align the releases with “good points in time” for desktop managers on unstable. Which constituted the most “in-house” work for 20.09. In this last release, plasma was the biggest pain, as we were shipping systemd 246, a mix of qt 5.12, 5.14, and 5.15 with plasma 5.18.5 which really shouldn’t exist together.
Shipping plasma 5.19 could have been an option, but it would have been EOL a week or two after the original target release date, and that’s awkward in case there is security issues, 5.18 is an LTS so we can rely on upstream for providing patches, but not for being compatible with things like the latest systemd. Moving the release date to november would mean that we could have packaged plasma 5.20, and kdeApplications on master before ZHF would start; and the 5.20 plasma release would have been supported for the majority of the 20.09 NixOS release. For gnome, we would just be lagging behind their releases by 4-6 weeks, which should be enough time for them to be packaged in nixpkgs.
A freeze period means certain changes won’t come in for a while which people running a “rolling distro” may expect.
cadence for staging is roughly 2 weeks when we aren’t “cramming”. It would a single ~2 week cycle skipped. PRs for bumps would still be allowed into master as long as they wouldn’t cause any regressions.
Make sense, thanks a lot for the clarification! For me any release time is good, even if it’s not always the same month depending on what’s practical for you. And thanks everybody for the great work, you’re amazing 
I know others have chimed in already, but I’d like to repeat that from my perspective shifting to NixOS 20.05 and 20.11 is essentially zero cost / downside, aside from the initial one-time churn and confusion. If @jonringer and other release managers think this simple shift will make a big positive difference I’d say we should do it. It sounds like there’s a lot of other stuff we can and should improve, too; but a simple +2m shift is a really easy hack.
We use NixOS for business (https://benaco.com) on the server and I use it on the desktop.
My opinions:
consul) does not support downgrading at all; after rollback, it will not start up. Lots of upstream software supports downgrading only a few versions. Stable OS releases help a lot here.