Tool for evaluating stateVersion changes

Goal: create a tool that would indicate if updating stateVersion is likely to break something.

Motivation: knowing if updating stateVersion will cause problems is at best tedious and at worst impractical for many users. It requires intimate knowledge of nixpkgs packages or options. The impact of changing stateVersion is implemented by conditional expressions in nixpkgs and should be testable by a program. Such a tool which indicates if an update is likely to break something would be very useful, particularly for new users [1].

[1] some advocate never changing stateVersion but not keeping it up to date with the current NixOS version seems inelegant. I want to know when changing it could break things. Even better, some indication of what would break as a hint where to focus attention is a worthy effort.

Approach: compare two “flattened” attribute sets [2] resulting from the evaluation of a configuration with stateVersion set to two different values. Any difference between the two indicates that changing stateVersion could result in different states and hence should be scrutinized more carefully. Conversely, if the two are identical, the output states should be identical and the user would know it is safe to update stateVersion.

[2] there is likely a better term for this but “flattening”

{{a = { b = 1; c = 2; }; d = { e = 3; f = 4; }; }

would return

{ a.b = 1; a.c = 2; d.e = 3; d.f = 4; }

Strawman: run the following twice with different values of stateVersion:

$ flatten-attribute-set <(generate-attribute-set --flake .#foo)  > flattened_config

then compare:

$ diff -u flattened_config.0 flattened_config.1

Any differences indicate that care should be taken, as well as point to the packages or services which could break. (Of course this logic should be encapsulated in a more elegant tool.)

Request: how can the attribute set of a configuration be computed and how can it be “flattened”? Specifically, how does one generate the “flattened” top level attribute set from a flake?

1 Like

“Keeping up do date” in what regard? The whole point of stateVersion is to decouple the NixOS version from the version of the state on-disk.

Sounds like a job for nix-diff.

1 Like

Will NixOS keep growing the list of conditionals ad infinitum? While this seems to be the current idea, I imagine that one day in the far future someone will want to clean up a bit… In such a case a tool like this would be useful. Or maybe we’re already beyond feaaibility for such an effort.

It would probably be useful just to check the delta, maybe some manual on-disk changes would make sense, but currently it’s untrackable.

Most services don’t react to stateVersion in any way; I see it as edge cases where you need to handle state this way. You can browse through git grep -l stateVersion nixos/modules

Nice. I never thought of using git grep -l .... Still that only gives a list of things that could potentially cause a problem, including things that aren’t used in the config and so they won’t cause a problem. A more newbie friendly approach would narrow down the list by excluding modules that aren’t used in the config.

I’d forgotten about nix-diff.

Is there an easy equivalent of nix-instantiate for flakes so nix-diff can compare the derivations? (This appears to be an open issue so I assume it hasn’t been solved yet.)

In the end, I built the flake using before and after settings for stateVersion. It made no difference with my configuration.

# nixos-rebuild build --flake .#mybox && mv result result.0
# sed -i.bak 's/stateVersion = 22.05/stateVersion = 23.05/' configuration.nix
# nixos-rebuild build --flake .#mybox && mv result result.1
# nix-diff $(nix-store --query --deriver $(readlink result.0)) \
           $(nix-store --query --deriver $(readlink result.1))  # no difference
# nixos-rebuild switch --flake .#mybox  # final confirmation; same generation

This is a bit more involved than it should be to answer the simple question “does changing it make a difference?”. I still think a tool is in order to abstract the details.

I don’t think newbies are meant to investigate this at all. They just keep stateVersion as it was and trust nixpkgs committers to do the right thing with it.

1 Like

Such is the design of stateVersion.
Adding decoupling after the fact is rarely elegant. If ever. Yet, that’s the purpose of stateVersion.

As a consequence of both observations, we should conclude that stateVersion is best avoided.
In fact, the darwin.linux-builder VM has been able to avoid even setting stateVersion at all.
It can get away with that, being stateless, and I hope we can keep it working without stateVersion.
HOWEVER, that’s a special case where we can afford to paint ourselves into a corner because the user can just dump the VM state. That’s not true in the general case, so you should set stateVersion in your configurations.
Just don’t rely on stateVersion unless absolutely necessary.

Other than that, I think stateVersion is underdeveloped. If we’re going to keep stateVersion around (and it looks like we will), I’d like to see a pattern where services declare that they use stateVersion and allow the stateVersion to be overridden in a per service option as well.
We have precedent for such cross-cutting options, such as assertions. However this one would not be accessed by top-level, but rather by a nixos-rebuild subcommand.
The data we currently have is just not sufficient, so some development is necessary.

Finally I hope we can replace stateVersion altogether by an abstraction that allows automation of the required upgrades, such as proposed in a recent RFC (although that RFC needs a lot of work, which makes me hesitant to link it. I just hope the authors get their work funded to improve it.)