I want to try out nixos-facter. I want to replace my hardware-specific option values in some configuration with usage of nixos-facter, and then examine the resulting differences. So I want the ability to compare two NixOS configurations.
Derivation-level Comparison
I did try using nix-diff on the derivations obtained via <my-configuration>.config.system.build.toplevel and of course it did show me the changes betweeen the two derivations. Which is useful and sometimes all one needs.
But what I really want in this case, is a higher level view. A view on the changes between the configurations. Well, at least thatās what I think I want. Itās hard to tell before I take a good look at such a comparison.
What is a configuration?
To the best of my understanding, a configuration is an evaluated set of modules. For example, a NixOS configuration is obtained in such a manner:
let
evaluated = nixpkgs.lib.nixosSystem {
modules = [
oneModule
anotherModule
];
};
in
evaluated.config
Kind of.
So just serialize them and compare
Using builtins.toJSON. Well, you canāt just do that, because:
This value also includes functions, for some reason. So these can be filtered away or mapped to strings.
This value also includes derivations. The attempt to serialize these results in building them. Okay, so map them to their .drvPath, instead.
Some values included are never evaluable, because the expressions simply always throw. For example, renamed options, I suppose. This can be worked-around using builtins.tryEval.
Weāre stuck at attempts to access non-existent attributes
The options for some values are meant to be accessed/evaluated only when some other option has some value. For example, foo.contents is meant to be evaluated only if foo.enable is true. To the effect that the value of foo.contents is an expression that would fail to evaluated with a missing attribute error otherwise. And missing attribute errors such as {}.a are not caught by builtins.tryEval, it seems. And I donāt see any other Nix language feature that can handle such an error.
How to proceed?
Does anyone else want to compare configurations? Or to serialize or just deeply evaluate a configuration for some purpose? Is Nixpkgs virtually full of these options that must not be evaluated unless some other option has some value? Are we missing something?
I was looking for the same thing when I was doing a major refactor of my config. I had to do it in really small steps and check if the results end up with the same hash.
Just stumbled across nixos-option in another thread. I had a quick check yesterday and ran into an issue with config.assertions but I think that is a bug in nixpkgs. In recursive mode the output should be diffable.
I have to play a bit more to really check if it fits my use case. It might even be a starting point for something else. I would rather have json output.
I see two obvious alternative paths toward being able to serialize configs:
Add a Nix builtin that is able to catch missing attribute errors and use that in the deep normalization of a config prior to comparison/serialization.
Attempt to make all ecosystem nixos modules attribute access safe. This could possibly be tested and the best place to start would be Nixpkgs but I suspect thoroughness might not be guaranteed with this approach. But many useful features have been achieved in this manner, so Iām not ruling this out.
currently there are two ways to deal with options which are optional: providing a default value/nullOr, or providing no default value and hoping no one wants to serialize them - the above mentioned PR could provide some value toward your goal here
A third possibility: add an unsafe builtin that serializes without forcing any more thunks, and use that builtin after evaluating system. The result would be only those configuration values that were actually used in evaluating the system, with no irrelevant attributes and no errors (assuming the system evaluates without errors, of course.)
A proof of concept: I added this tiny bit of code to src/libexpr/primops.cc:
static void prim_unsafeToXMLNoEval(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
std::ostringstream out;
NixStringContext context;
auto arg = *args[0];
state.forceValue(arg, pos);
printValueAsXML(state, false, false, arg, out, context, pos);
v.mkString(toView(out), context);
}
static RegisterPrimOp primop_unsafeToXMLNoEval({
.name = "__unsafeToXMLNoEval",
.args = {"e"},
.doc = R"(
As `toXML`, but without forcing any thunks. The result will depend on how
much of the argument has already been evaluated.
)",
.fun = prim_unsafeToXMLNoEval,
});
Now I can run this to get a relatively diff-friendly XML dump of my config:
Finding or writing an XML diffing tool that compares these dumps for a more user-friendly presentation of which attribute paths changed and how is left as an exercise for the reader.
I made some change to nixos-options to make it print JSON, but it is too slow to evaluate all configuration.
It consumes more than 30 GB of memory, and then OOM killed. Evaluate one sub-option at a time, then merge the results might be a solution.
Following the same idea, I added another builtin, builtins.anotherToJSON, to print JSON, support thunks and functions, and avoid forcing any more thunks. Itās great!
Any ideas how to make this work with flake? Flake doesnāt seem to have a system attribute, without it, the JSON value is all thunks.
To run it with Flake (using home manager may cause an OOM issue; see below):
build/src/nix/nix eval --show-trace --raw --impure --expr 'let
os = (builtins.getFlake "git+file:///<path-to-flake>").nixosConfigurations."<your-os>";
config = os.config;
configToEval = config;
system = config.system.build.toplevel;
in
builtins.seq system (builtins.anotherToJSON configToEval)
' > config.json
The cause of the OOM issue is home-managerās extraSpecialArgs. It refers to os.config, which in turn refers to home-manager, creating a circular reference. Since I donāt need to access osConfig in home manager, I set it to {}, which resolves the OOM issue.
The OOM issue in my modified nixos-option is also caused by this circular reference.
After excluding assertions, meta, services, system, virtualisation and home-manager from the config, nixos-option can finally print JSON without throwing any errors.
During this tweaking, I found that many errors were caused by accessing non-existent attribute, for example, trying to access a missing maintainer, inheriting a non-existent attribute, etc. These are human errors and could be fixed.
Maybe we could add a test to evaluate all attributes in nixosConfigurations to check if they throw an attribute '' missing error and possibly identify which file is causing the issue.