Potential RFC Idea: Stability
This is an idea for an RFC we might want to have. It originated from discussions around https://github.com/NixOS/rfcs/pull/136 between me and @Ericson2314.
The idea is to establish some basic principles for the stability lifecycle of features in Nix, similar to the Linux Kernel’s ABI Readme.
This already pretty much exists for Nix itself here. The scope of this RFC may be more generally for all of Nixpkgs, Nix and NixOS.
Stability levels
All features should go through this lifecycle:
-
Experimental
- Requires explicit opt-in
- May be changed in breaking ways or removed at any time without informing users
A feature may become stable if:
- Relevant stakeholders are confident in its design
- The feature is reasonably well tested by CI
- The feature is reasonably well documented
When a feature becomes stable, the minor version must be increased, see semver.
-
Stable
- Intentional breaking changes are not allowed:
- The CI tests and for this feature must continue working as-is
- The documentation for this feature must continue to be accurate
- Unintentional breaking changes are bugs and should be fixed
- A CI test case should be added to ensure the bug can’t happen again
A feature may become deprecated if:
- Relevant stakeholders decide it shouldn’t be used anymore
- All the use cases of the feature can still be fulfilled
When a feature becomes deprecated, the minor version must be increased, see semver.
-
Deprecated
- Intentional and unintentional changes are still not allowed, same as for Stable.
- Using the feature must throw a warning message that includes:
- Replacement(s) for the use cases of the feature
- Optionally when the feature will be removed or changed in a breaking way
A feature may be removed if:
- The agreed-upon deprecation period has passed.
When a feature gets removed, the major version must be increased, see semver.
-
Removed
- The functionality doesn’t exist anymore
Note that not all features go through every step:
- Simple feature can instantly become stable and skip being experimental
- Features that we have no problem with need not be deprecated
Stable features may not depend on unstable features.
Timeline
Lay out a timeline, describing how e.g. 2 years have to pass before a deprecated feature may be removed.
6 Likes
Yes I think this is very good idea. Hopefully the basic “experimental → stable → depecated → removed” lifecycle is uncontroversial; the details might be but we can adjust how rigid or flexible we want to be based on feedback.
Very recently we added https://github.com/NixOS/nix/blob/91856396317995aa38dc7244357596b8de27f937/doc/manual/src/contributing/experimental-features.md to the Nix docs, which started explaining this life cycle, but just the beginning parts. “stable” is not defined (we very intentionally just use and define unstable, not knowing what exactly the other side), and not mentioning “deprecated” at all.
Also, that document naturally only is about Nix itself. An RFC has the virtual of being able to nail down these principles for the entire ecosystem, only secondarily worrying about how they apply to any one individual process. (e.g. formal stability for Nixpkgs is hard because have no facility for abstract data types*, which means what is intended to be private implementation details often leaks).
I feel like this is just stating the obvious, for which we have documentation, and not RFCs. RFCs describe changes and are ill-suited as documentation, as the audience wants to read about the end state.
@fricklerhandwerk do we have a place for overarching reference documentation? A document like this applies to the whole ecosystem.
It gets more interesting when you look at how these stages interact with the requirements of each individual project. For instance, Nix can hardly ever deprecate or remove anything, because that tends to break old expressions, which we would like to keep supporting.
I think you could add Stable → Discouraged → Deprecated.
That would give us an end state for things that aren’t worth removing, but would be omitted if we could start over without throwing away valuable legacy somehow.
1 Like
Removed can be further refined to Removed → Disappeared, where Removed throws an error message explaining the removal and alternatives if any.
Experimental
A feature may become stable if […] The feature is reasonably well tested by CI
Testing must be a merge condition, because there’s realistically, there’s no way to get good tests after sufficient time has passed for the ecosystem to validate the feature.
In other words, tests are part of the development cycle, and not part of the stability cycle.
1 Like
No, and I don’t think that would be a good idea. RFCs already have the problem that they only implicitly require code owners‘ participation, and imposing such a structure on each project from the top is not something I would want have or could imagine happening easily.
For this specific case, I’d rather encourage maintainers adopting such a template, and an RFC may be meaningful to provide a suggestion, recommendation, or guideline; not state a rule that can’t be enforced anyway.
For instance, I’d love to have (a slight variant of) what’s in the top post for Nix, but I wouldn’t want to mess with how Nixpkgs or NixOS maintainers deal with stability.
Currently the place for documenting what one could say are global properties of the ecosystem is nix.dev, and there aren’t that many. Also note that we’re documenting what is, not what could or should be.
If the goal is to “establish principles” then to me that sounds more like establishing a common set of terminology than enforcing a common set of rules.
I guess my problem with RFCs is that it’s unclear what they produce. Some propose a change, some provide out of date reference documentation, and it also seems to be ok for an RFC to shout into the void.
So we can’t have expectations about an RFC, and that makes them unsuitable as documentation, except as a paper trail for whatever changes happened, if they happened.
1 Like