But the migration would fail. Would that failure reach the user in a sensible fashion?
Automated migrations are almost always fraught with ugly potentialities… and that’s one of the reasons we have stateVersion.
Well, that’s a completely different point. I was personally only commenting on the “mv is not atomic” comment, not the issue of whether migrations as a whole are feasible. But, IMO, without going into detail, failures in edge cases like that IMO are acceptable, and typically it’s reasonable to make the failure pretty obvious. Like in this case the migration would probably simply happen if it can easily, or fail the service if it can’t. Obviously ExecStartPre=-mv ... doesn’t fail the service, but again I think this example was just off the cuff and not what would be used in reality.
1 Like
But even in such a simple case, we’ve already discussed a number of naive implementations that wouldn’t have the right characteristics. Could you get those things right if you were writing the module? Probably. But how many of the people writing and reviewing module changes are as good at this as you or me? Is “probably” really good enough considering the potential of catastrophic data loss? (Yes, everyone should have backups. Lots of people don’t. “NixOS ate my data” is a really bad thing for people to be legitimately saying, regardless.)
If it can only be done robustly by super-programmers, it’s not a reasonable thing for nixpkgs to force module writers to do. Migrations are just inherently risky in this sense, since bugs in the process fundamentally can irreversibly destroy things far more easily than almost anything else we might write.
2 Likes
Idk mv /var/lib/{jellyseerr,seer} in ExecPreStart for everyone seems like a bad idea in general. Sometimes after upgrading you notice that enough stuff is broken that you have to rollback to the previous build but now you can’t do that easily because some stateful scripts messed with the system. Just tell the users to set a homeDir in the config when you asked them to rename the service config. Maybe even make the option required for a release cycle.
4 Likes
Maybe even a switch-time sanity check that refuses to switch if a jellyseer directory is found, plus a break-the-glass option? I do agree that there are probably less risky ways to handle these types of state changes.
This all seems a bit off-topic, though, this is a specific edge case. Better to discuss in a PR that tries to blanket-undo these types of uses of system.stateVersion, or at least this specific one?
This is why I like @ElvishJerricco 's initial suggestion, though. It’s a quick win in terms of useability, and clearly marks services which might need some thinking to be able to deprecate system.stateVersion.
Maybe it just causes a rush of people thinking about how to avoid adding these extra options to various modules to prevent that change, but that’d be ideal.
1 Like
stateVersion or a moral equivalent is clearly necessary as long as there is on-disk state to manage; automagic migration is clearly a bad idea to even attempt outside totally trivial cases.
instead of deprecating the mechanism I’d prefer for there to be a reliable way to query “which modules are going to be affected if I change stateVersion to be x.y" (but I’ve no idea how to do that, of course).
2 Likes
To be clear, my feeling on that idea is that it’s useful when stateVersion as a concept is actually necessary. But I think @emily has done a good job explaining that the real-world cases in nixpkgs where it’s necessary are very few (possibly zero), and that the alternatives (at least in most cases) are comparably difficult to implement.
3 Likes
This thread has meandered a lot, but this is one of the problems the top-level proposal directly addresses, specifically with the (computed, not user-written) system.moduleStateVersions option. The idea is that you will be able to compare the value of that option before and after changing system.stateVersion in order to see which modules are affected, and then you will have the option of migrating the affected modules incrementally by setting each individual module stateVersion instead of having to do them all at once by actually committing the system.stateVersion change.
(A relative disadvantage is that, while the benefits of being able to migrate individual modules will be available to those modules as they incrementally adopt this pattern, the benefit of being able to assume system.moduleStateVersions is comprehensive would only be realized if every use of system.stateVersion goes through this pattern, which will take longer if only because a larger proportion of high-influence committers will need to be convinced in order to make this a project-wide requirement.)
got me. I was making to many assumptions that come from my own systems.
But with the big if that it cannot be cross moinpoints it just feels to risky to generally do and I would not like it in review.
7 Likes
Could this maybe be useful to figure out what would change? Diffing NixOS configurations at the config-level