It’s not quite as simple as that. Or at least you’re suggesting that a whole lot can be checked without any extra input, which just isn’t the case.
Your idea seems to assume that all dependencies are provided by NixOS itself and not some other flake as well. (See hermeticity note below)
For instance, a NixOS module may only be valid in the context of the presence of some other optional module, such as the NixOps module for secrets deployment. That’s still a valid NixOS module, and certainly not a NixOps module because that would refer to the whole network and not just a node.
Also the module system does not have a concept of which parts of the option tree are meant to be valid. It is a lazy configuration system, and we often rely on that laziness, both for correctness and for performance.
system.build.toplevel is only an approximation, as images may still fail to evaluate as they are not contained within that attribute.
And even than, evaluating that isn’t good enough, because more often than not, the module has to be “enabled” by defining some config.
To check for correct evaluation of a module, we have to rely on tests. Evaluating an example is a more reliable sanity check than somehow evaluating a module in the absence of a user configuration anyway, so it isn’t all that bad.
And an external NixOS test would of course make it complete.
What we could do is a very very basic check of the imports, but for anything else, for this to still work in the general case, we need an example anyway. Still not sure if even that is a good idea, because that would set expectations for imports in submodules, for which we again need an example and an attribute path to actually find them.
So to make such a check actually work we need to generalize the schema feature to just general evaluation checks; the backbone of a testing framework (ie the basic structure, but not one of those custom assertion libraries, except perhaps for something that represents exceptions which is not otherwise possible within the Nix language).
Defining evaluation checks as part of the schema seems like we’d be getting the features backwards. (Might be a pattern with flakes, and I’d like for development to be a bit more first principles again.)
Maybe something could be done by introducing more meta options, such as a meta.examples.<name>.{modules,optionPaths} so that users can define such checks right inside their module instead of in some other flake attribute. That would be a nice module to have as, say, nixpkgs.modules.generic.examples (in accordance with my suggestion in nix#6257)
Hermeticity and coherence
We currently can’t have both hermeticity and coherence in the module system in a standard way. By importing all dependencies, you will run into duplicates due to diamond dependencies in the flake inputs graph. If you pick coherence instead, you have to rely on the “end user” of the modules - ie the configuration - to be aware of all the imports transitively. I can’t even think of how this impacts the imports in submodules pattern right now, so it’s all just a bit sad in this area.
We could make additions to the module system to at least help out with those coherence / diamond dependency errors, reporting them as such, instead of arbitrary erros relating to all sorts of collisions, but to make this work, we require such things as flake names, widespread use of flake frameworks, or a bit more boilerplate when distributing modules. To be continued.
. Even though I’m not really a flake user, I felt that having the schema there was a bit too much detail for a core feature of Nix. Well done!