Incrementally salvaging stateVersion

[nyanbinary]

Do we really need to bikeshed on an RFC for multiple years I do like how in recent times things were first proposed to nixpkgs rather than bikeshedding on an RFC.

[TLATER]

I think there are some no-brainers, like moving all modules for software that doesn’t want to skip versions on update to not having a default for their .package option.

But there are other approaches to state handling that aren’t as straightforward, and are more policy discussion than raw code, like making NixOS also not like it when you skip past versions. I think those approaches should go through an RFC simply because they are big changes to how you’re supposed to write modules and I think module authors deserve a chance to give their opinion.

[withakay]

In https://nixos.org/manual/nixos/stable/release-notes#sec-release-26.05:

services.stalwart-mail has been renamed to services.stalwart to align with upstream re-brand as an e-mail and collaboration server. Other notable breaking changes to module:

• Addition of module-specific stateVersion option, which on existing installations of Stalwart must be set to the same as system.stateVersion.This enables manually and carefully migrating Stalwart to a new stateVersion or newly enabling the Stalwart module with a newer stateVersion than system.stateVersion.

Seem like it an implementation of the base idea enumerated in this thread, so it’s pretty cool to see this experiment happening in tree.

One difference (of potentially trivial import) is that the format of services.stalwart-mail.stateVersion is a version string (“26.11”-like), whereas the proposal here seems to have been integers.

[TLATER]

@withakay

One difference (of potentially trivial import) is that the format of services.stalwart-mail.stateVersion is a version string (“26.11”-like), whereas the proposal here seems to have been integers.

I think that’s a shame; the version strings are really hostile to users of the unstable channel, because you effectively can never safely introduce breaking changes.

It’d be nice if future experiments do away with that, though I appreciate this is probably to make defaulting to system.stateVersion easier.

[withakay]

On reflection, services.stalwart-mail.stateVersion represents something of a departure from the proposal in the OP in more significant ways.

Specifically, the OP proposes that

[module.stateVersion] should take a default value that depends on system.stateVersion

whereas services.stalwart-mail.stateVersion appears to require an explicit definition for the configuration to evaluate. If anyone is actually using stalwart-mail, it would be helpful to know if this is correct and 26.05 did require an explicit stateVersion (thus breaking a pre-existing config)

@TLATER

I think that’s a shame; the version strings are really hostile to users of the unstable channel, because you effectively can never safely introduce breaking changes.

Requiring explicit definition should allow for breaking changes to be introduced regardless of int vs version string, since users will have to explicitly change module.stateVersion to opt-in to the breakage. I think?

I think version strings do have the limitation that they must be of form “YY.MM” though, which may eventually scale poorly if (for some horrible reason) there is a need to introduce 2 different breaking changes in the same month. There is also some theoretical issue with distributing a Y2K-like problem across multiple modules.

services.stalwart-mail.stateVersion also does not implement the system.moduleStateVersions."services.stalwart-mail" pattern discussed in the OP, which I think was intended to help make module.stateVersion changes more discoverable. In a way, it feels like services.stalwart-mail just skipped over some of the parts that were intended by the OP to make a treewide more incremental and prevent existing config breakage.

[NobbZ]

@withakay

Requiring explicit definition should allow for breaking changes to be introduced regardless of int vs version string

Once allows them at any point in time, the other only once every six months.

[withakay]

@NobbZ

@withakay

Requiring explicit definition should allow for breaking changes to be introduced regardless of int vs version string

Once allows them at any point in time, the other only once every six months.

I think I addressed this with:

@withakay

I think version strings do have the limitation that they must be of form “YY.MM” though, which may eventually scale poorly if (for some horrible reason) there is a need to introduce 2 different breaking changes in the same month.

And I may have actually been somewhat wrong about even that limitation (1/month):

nix-repl> pkgs.lib.versionAtLeast "26.05.01" "26.05.30"
false

nix-repl> pkgs.lib.versionAtLeast "26.05.30" "26.05.01"
true

nix-repl> pkgs.lib.versionAtLeast "26.05" "26.05.30"
false

[withakay]

Strictly within the context where a user must define an explicit module.stateVersion:

In light of this, it might be arguable that the ergonomics of module.stateVersion as a date-like version string are actually better. Whenever a module is first added to a configuration, a date-like version string permits entering the current date (on unstable) or the “YY.MM” of the current stable release (on stable). With an integer, they must read the source or documentation to determine what the latest module.stateVersion should be.

[rhendric]

@withakay

Strictly within the context where a user must define an explicit module.stateVersion

There’s a fair amount of drift here from my original proposal, but maybe that’s okay; we can narrowly discuss its merits in the PR now.

But since we’re here, I’ll just say on record that I rather dislike the idea of requiring users to define module stateVersions always. Having a single configuration-wide default is good; the bad things that currently come with it can be corrected without requiring every user to add clutter to their configs if they don’t want it. (Users that want to live in this world are welcome to it, of course.)

[frank87]

@emily

(If we had editions/actual migration tooling, then it would be simple to handle: we switch the default, and people migrating from an earlier edition get systemd.services.the-service.serviceConfig.StateDirectory = "/var/lib/old"; set explicitly in their configuration. But as I said, I don’t think we need this to handle the existing stateVersion uses, and don’t propose anyone block on it.)

What would migration tooling look like? I suspect it would be some tools to help upgrading safely(backups, blocking dependent services etc). In the end, the package has to manage it’s own data. We can’t do that on the OS-level.

Realizing that, changes the problem definition: what can we do to help packages handle mutable data from different versions. It looks more like an additional tooling-package than an OS update.

(You explicitly don’t want to rollback mutable data: that’s lost production)

[TLATER]

@frank87

What would migration tooling look like?

Have a peek through the related posts, this has been discussed at length, and it’s mostly tangential to this post.

In a nutshell, though, most uses of stateVersion aren’t related to data migration at all, but simple things like directory renames. Most uses aren’t even related to state and simply pick default versions.

The argument hence goes that, in theory, most use cases of stateVersion seem to be pretty easy to eliminate even just by policy (e.g. “don’t rename state directories”), and the few remaining ones could be handled like postgres, nextcloud, kanidm, etc., forcing users to explicitly state the package version and handle migration themselves.

But anyway, there is plenty of discussion around this already, no real use in repeating all that without someone sitting down and writing a clear proposal on how that would be done.