Flake schemas: Making flake outputs extensible

Hi all,

I’ve just created this PR: Flake schemas by edolstra · Pull Request #8892 · NixOS/nix · GitHub. It proposes a feature called flake schemas.

A big problem with current flakes is that commands like nix flake check and nix flake show have built-in support for a limited set of flake output types. For instance, they know about nixosConfigurations but not homeConfigurations.

The flake-schemas PR moves support for flake output types out of C++ and into Nix functions supplied by a new schemas flake output. This allows users to define their own flake output types, and have them show up in nix flake show and checked by nix flake check. As a result, there are no more flake output types that are more “blessed” than others.

There is no central registry of schemas. Instead a flake should add an appropriate schemas flake output for the outputs it wants to have checked. Typically this will include schemas from other flakes. In particular, I’ve made a flake-schemas flake with schemas for the flake output types that were previously built into Nix.

I’ve written more about flake schemas on the Determinate Systems blog. Feedback is very welcome!

34 Likes

That’s an interesting feature that aligns with some ideas that have been discussed in the Nix team. It would have been a good idea to discuss the design with the team first, but of course we’ll have a look at the PR.

16 Likes

Happy about this :smiley:. Even though I’m not really a flake user, I felt that having the schema there was a bit too much detail for a core feature of Nix. Well done!

5 Likes

This is a very nice feature. I saw the post about it and I was interesting if this will tie into nix build so things like Home Manager, nix-darwin, and NixOS Configurations can specify the derivation to use (ex: nix build .#nixosConfigurations.nixos which builds nixosConfigurations.nixos.config.system.build.toplevel).

4 Likes

I wonder how this will play out with nixpkgs, since the general policy is not to have flake dependencies.

Flake outputs can depend on other outputs of the same flake by using the self input, so a flake can define its own schemas. This is, indeed, what flake-schemas does.

Since this is a proposal, it makes sense for the schemas to be defined somewhere else, but if there is appetite for this, we could consider adding the schema validations to nixpkgs directly, in which case it will contain its own schema validations without any dependencies.

3 Likes

Yes, the nixpkgs repo would be the logical place for the canonical schema definitions of the overlays, nixosModules and nixosConfigurations schemas. In fact, that allows evaluating overlays and nixosModules against an actual Nixpkgs/NixOS, which is currently impossible (since nix flake check doesn’t have access to the right nixpkgs flake).

7 Likes

It’s not quite as simple as that. Or at least you’re suggesting that a whole lot can be checked without any extra input, which just isn’t the case.

Your idea seems to assume that all dependencies are provided by NixOS itself and not some other flake as well. (See hermeticity note below)
For instance, a NixOS module may only be valid in the context of the presence of some other optional module, such as the NixOps module for secrets deployment. That’s still a valid NixOS module, and certainly not a NixOps module because that would refer to the whole network and not just a node.

Also the module system does not have a concept of which parts of the option tree are meant to be valid. It is a lazy configuration system, and we often rely on that laziness, both for correctness and for performance.
system.build.toplevel is only an approximation, as images may still fail to evaluate as they are not contained within that attribute.
And even than, evaluating that isn’t good enough, because more often than not, the module has to be “enabled” by defining some config.

To check for correct evaluation of a module, we have to rely on tests. Evaluating an example is a more reliable sanity check than somehow evaluating a module in the absence of a user configuration anyway, so it isn’t all that bad.
And an external NixOS test would of course make it complete.

What we could do is a very very basic check of the imports, but for anything else, for this to still work in the general case, we need an example anyway. Still not sure if even that is a good idea, because that would set expectations for imports in submodules, for which we again need an example and an attribute path to actually find them.

So to make such a check actually work we need to generalize the schema feature to just general evaluation checks; the backbone of a testing framework (ie the basic structure, but not one of those custom assertion libraries, except perhaps for something that represents exceptions which is not otherwise possible within the Nix language).

Defining evaluation checks as part of the schema seems like we’d be getting the features backwards. (Might be a pattern with flakes, and I’d like for development to be a bit more first principles again.)

Maybe something could be done by introducing more meta options, such as a meta.examples.<name>.{modules,optionPaths} so that users can define such checks right inside their module instead of in some other flake attribute. That would be a nice module to have as, say, nixpkgs.modules.generic.examples (in accordance with my suggestion in nix#6257)

Hermeticity and coherence

We currently can’t have both hermeticity and coherence in the module system in a standard way. By importing all dependencies, you will run into duplicates due to diamond dependencies in the flake inputs graph. If you pick coherence instead, you have to rely on the “end user” of the modules - ie the configuration - to be aware of all the imports transitively. I can’t even think of how this impacts the imports in submodules pattern right now, so it’s all just a bit sad in this area.
We could make additions to the module system to at least help out with those coherence / diamond dependency errors, reporting them as such, instead of arbitrary erros relating to all sorts of collisions, but to make this work, we require such things as flake names, widespread use of flake frameworks, or a bit more boilerplate when distributing modules. To be continued.

1 Like

If you were trying to refer to Nixpkgs lib, just for the module system to confirm that the modules are attrsets or paths in an architecturally perfect way, that is of course within reach.

Also don’t worry too much about the module system. It is a rather unique system thanks to laziness and runtime checks, and we’re in the (admittedly slow) process of documenting it.

1 Like