Thoughts about Improving NixOS Release Notes

Meta
#1

As mentioned in the 21.11 Retrospective, release notes were fairly neglected. Generally it fell on a release manager to do release notes, but all of the release managers were per-occupied with ZHF, heated RFC 98 discussion, and their personal lives.

Main issues:

  • Disorganized: didn’t seem to be much a logical ordering to release notes. Maybe have categories for related technologies?
  • Lack of prioritization: Highlights, which aren’t highlights. Hard for users to “find relevant” items.

Proposed solutions:

  • Release editor Role (21.11 retrospective): In addition to having a release manager, we could also have a release editor; solely focused on making release notes easier to digest.
    • Also, a different (non-technical) skillset. May be a good position for someone who wants to be involved with NixOS (or FOSS), but doesn’t want to focus on technical issues.

I was curious how others felt about broadening the release related roles. Or if there’s other potential solutions.

9 Likes
21.11 Retrospective
#2

Regarding organization:

The simplest is to reflect the Nixpkgs folder structure. It’s not perfect, but is at least an existing way to decide what goes where. After that, one could imagine a section for breaking changes and another for non-breaking notes.

1 Like
#3

Why not have stronger requirements for release notes on pull requests? Nothing too strict or objective; just if the person pressing the merge button thinks release notes are appropriate, then consider that a blocker for the PR (no matter who actually goes in to write them). Something similar would probably be appropriate for documentation.

#4

I don’t think a 1-to-1 is appropriate

applications   common-updater  desktops     games  os-specific  servers  stdenv  tools
build-support  data            development  misc   pkgs-lib     shells   test    top-level

maybe combination with nixos options:

config.appstream          config.gnu                config.nix                config.specialisation
config.assertions         config.gtk                config.nixops             config.swapDevices
config.boot               config.hardware           config.nixpkgs            config.system
config.console            config.i18n               config.passthru           config.systemd
config.containers         config.ids                config.power              config.time
config.docker-containers  config.jobs               config.powerManagement    config.users
config.documentation      config.krb5               config.programs           config.virtualisation
config.dysnomia           config.lib                config.qt5                config.warnings
config.ec2                config.location           config.security           config.xdg
config.environment        config.meta               config.services           config.zramSwap
config.fileSystems        config.nesting            config.snapraid
config.fonts              config.networking         config.sound

Maybe just take some inspiration from other distros:

#5

There was something added to the PR template to ensure that people are adding a release note entry. pull_request_template: Add release note criteria · NixOS/nixpkgs@291dcac · GitHub

But I would argue that someone still needs to do some global prioritization, organization, editing, and groooming.

2 Likes
#6

Technical writing is a vital skill, too often undervalued by our industry. I think a release editor could do wonders for NixOS.

11 Likes
#7

For the ordering, I’d like to have it alphabetically. Our “append to the end” approach lead to a lot of conflicts and rebasing with slow pull requests, which I found rather tedious to resolve. An alphabetical sorting would greatly reduce the risk of collisions, similar to how rarely pull requests collide in all-packages.nix. I don’t know whether keeping the alphabetical order for the actual release notes is a good idea or not. Maybe it is, because it makes finding information about packages you are about easier.

I’d be in favor or a release notes editor (or team). However, simply appointing somebody responsible is not enough: we must also give them the power to actually do things. For example, we will probably need to implement some “release notes freeze” period before the release.

3 Likes
#8

IDK about a literal 1-to-1, but I guess one ~theoretical advantage to notes that are closer to either the source or attrset hierarchy is that it could make it easier to someday automatically filter a list of notes that pertain to a system/profile/build/shell? (I imagine it could drive UX improvements like a ~preflight for nixOS updates and maybe an ~early-warning message about accumulated/pending release notes.)

#9

sure, but we should probably curate how the nixpkgs/pkgs folder is structured then. And I would suspect that you would still want someone to ensure that the release notes convey the right meaning.

#10

I think if the release notes were more structured, then it would be more apparent to contributors how they release notes should look and where they should be placed.

I think the main issue now, is that there is no rhyme or reason, so people just slap something there.

1 Like
#11

I would prefer an alphabetical sorted list of items.

Each release note item then can be tagged accordingly, with at least one tag out of [ "main:highlights" "main:backwardIncompatibilities" "main:newService" "main:notableChanges" ] and garnished with additional tags [ "config:nix" "..." ].

Release Manager than can structure release notes based on that tagged list.

#12

I think this is a great idea.

In addition to allowing us to access a different skill-set pool, it could also serve as an on-ramp to becoming a RM.

For this role, I would suggest we waive the requirement (which is present for RMs) to have a nixpkgs commit bit.

This could allow someone to dip their toes in the RM process, and maybe later consider becoming a RM, having already been involved first-hand.

Even if this doesn’t happen, technical writing is a wonderful skill in itself, and we should cultivate it :heart:

1 Like
#13

This isn’t strictly release notes, but it would be pretty amazing if NixOS had a tool I could run that would tell me what changes affect any module that I have touched in my config (using the definition locations to determine what’s been modified from outside of the NixOS tree). Determining package-level changes that affect any package in my closure would be nice too; that’s less obviously correct (e.g. changes to an indirect dependency may not necessarily affect me), though things like “please back up your PostgreSQL before updating” would still be good to know even if that’s an indirect dependency.

This wouldn’t tell me everything I care about (e.g. new modules or enhancements to existing modules may encourage me to add those to my config) but it would be a great way to avoid missing anything.

2 Likes
#14

but it would be pretty amazing if NixOS had a tool I could run that would tell me what changes affect any module that I have touched in my config (using the definition locations to determine what’s been modified from outside of the NixOS tree).

First impressions are that this isn’t possible at the nix or nixpkgs level. It would have to be at the nixos (or nix-darwin) level; using something like /run/current-system/ to determine a diff. And even then, nix is more about, “give me some end state, given some inputs”, not “give me some end state with upgrade output, given some system input, and the current system”.

I mean, it could be done in theory for NixOS modules. There could be an optional specialArg, which points to the serialized form of the previous config json value; that value then can be read while doing the evaluation for the next generation. Modules would just import a { lib, pkgs, config, activeConfig, ... }, and then comparisons between the desired endstate of config, and the current reality of activeConfig could be asserted. Known failure states could abort quickly, and the operator would need to reconcile the difference. Suggestions could just emit warnings, similar to the mkRemoveOption that we have now.

But, in practice I think it would introduce a lot of complexity, additional evaluation time, and maintenance effort for being marginally meaningful. Only super stateful services such as databases or nextcloud would really benefit from something like this. And the upgrade paths for these services are generally non-obvious. I guessr the nextcloud module has attempted to do this as nextcloud only supports upgrades from the previous major version, however, they were using stateVersion to trace the starting point, and that’s not really the intended use case for stateVersion.

1 Like
#15

I was thinking about this as being at the NixOS module level. But it wouldn’t even require evaluating against the new version. You could have a script that walks the options tree, checks the definition locations for every option, and any time it finds a definition that’s not in the NixOS tree it spits out the option attribute path. You can then take the list of all changed modules from the release notes and compare them to the list of modified config options and see which options paths are prefixed by a modified module.

I’m not sure just how useful this would be in practice, but it does seem as though “what modules that I use are updated in this release?” is a good question to be able to answer.

This approach doesn’t help with “what packages that I use have non-trivial updates?” but I’m sure something could be done by looking at the system closure.

#16

I opened a PR structuring the release notes a bit more. Hopefully people will have a better idea how and where something should be inserted.

Hosted by Flying Circus.