Introducing the NixOS to JSON Schema Converter

Announcements
1
25 Likes
2

Did we just accidentally implement a LuCI interface for NixOS?

4 Likes
3

love seeing this, props on your progress!

there’s a need to configure these modules from diverse frontends, such as:

i’m missing TUIs from that list :smile: (disclaimer: i’m not aware of any existing tui-jsonschema-form)

1 Like
4

That would be amazing, but I don’t understand how. I’m not too familiar with LuCI. Could you elaborate?

5

Sorry for the late reply, I was preparing for an exam the past days. LuCI is a settings panel developed as a basic component of OpenWRT, which basically consists of a list of settings (Click to select, drop down, input box, etc. things we can normally see in the settings interface), and some information such as IP address and load statistics.

For settings part, see https://openwrt.org/docs/guide-user/base-system/uci

1 Like
6

This is dope!

I started trying to implement something similar, but in the opposite direction (JSON Schema → Nix option declarations), but I’ve been stuck trying to figure out how I want to handle meta-schemas, since accepting arbitrary JSON Schema input means I’ll have to collect the entire closure of all the meta-schemas and vocabularies reachable by any schema that might be fed as input. That and handing meta-schema specific keywords like $dynamicAnchor and $dynamicRef.

This blog post and implementation has me thinking I may be taking the wrong approach and maybe I should try to get an MVP working that ignores all the meta-schema shenanigans to focus on just the structures common to a broad set of non-meta schema documents.

If anyone has any insights on JSON Schema or useful resources that would help with parsing all the references, I’d greatly appreciate it! Kinda kicking myself now for not taking a compilers course in school.

7

oo, what use-case did you have in mind for this?
focusing on the commonly used structures def seems fair! i know json schema can get tricky with stuff like anyOf / allOf…
for resolving of json schema references from python i’d been using jsonschema-path, tho other languages should have their own tools.

2 Likes
8

JSON schemas cover a lot more stuff than Nix modules, so if implemented, we’d get a massive increase in the number of things we can interface with through Nix.

The main idea is essentially allowing the trivial creation of Nix modules from JSON schema definitions.

A basic implementation could look something like this:

{ config, lib, pkgs, ... }: 
let
  cfg = config.programs.my-program;
in
{
  options.programs.my-program = {
    enable = lib.options.mkEnableOption "Enable my-program";
    package = lib.options.mkPackageOption pkgs "my-program" {};
    settings = lib.options.mkJsonSchemaOption {
      path = ./some-program.schema.json;
      hash = lib.fakeHash;  # Hash of closure of all schemas.
    };
  };
  config = lib.mkIf cfg.enable {
    environment.etc."my-program.json".text = builtins.toJSON cfg.settings;
    # ... rest of config ...
  };
}

…where the settings option gets validated against the JSON schema specified inside mkJsonSchemaOption.

Users would basically specify a path / URI to a JSON schema definition. Starting from there, it would recursively find all other JSON schemas referenced in $schema or $vocabulary fields.

To maintain reproducibility, we'd need to do one of the following:
  1. pin the closure of referenced docs to a hash.
  2. pin each individual referenced doc to their own hashes.
  3. combine all referenced docs into a completely standalone schema doc and pin to its hash.

If Nix modules had first-class JSON schema support in nixpkgs, we could do some cool auto-magic stuff.

Auto-generate NixOS/HM modules from package definitions

Since upstream projects often provide JSON schemas in the same repo as their source code, we could collect schema.json files added to $out/share/schemas like we currently do with .desktop files in $out/share/applications. The default schema could be specified like how meta.mainProgram selects executables from $out/bin

stdenv.mkDerivation {
  meta = {
    mainProgram = "my-prog";
    runtimeConfiguration = {
      mainSchema = "my-program.schema.json";
      pathSuffix = "my-program.json";
    };
  };
};

then we could have a couple basic libs like lib.getExe & lib.getExe' that use these to simplify option declarations. e.g.

{ config, lib, pkgs, ... }@moduleArgs: 
let
  cfg = config.programs.my-program;
  isHM = config ? "home";
in
{
  options.programs.my-program = rec {
    enable = lib.options.mkEnableOption "Enable my-program";
    package = lib.options.mkPackageOption pkgs "my-program" {};
    settings = lib.options.mkOption {
      type = lib.types.jsonschema (lib.getConfigSchema package);
    };
  };
  config = lib.mkIf cfg.enable {}
    // lib.optionalAttrs  isHM {  xdg.configFile = lib.getConf cfg.package; }
    // lib.optionalAttrs !isHM { environment.etc = lib.getConf cfg.package; };
}

for basic programs that read a config file, we could have a simple wrapper lib to generate all of this.

This has a number of benefits:

  • Improves existing settings / extraConfig options with schema validation.
  • Eliminates the need to update Nix modules when new versions of the upstream package change the config schema.
  • Eliminates the need to manually create new modules for simple programs that provide a schema definition.
  • Simple modules could be shared between both NixOS & home-manager without having to write similar, yet incompatible module definitions.
Use the Nix module system to build a huge variety of artifacts

Basically leverage lib.evalModules to use Nix’s templating and merging capabilities to build thousands of new artifact types.

schemastore.org has (at the time of writing) 924 JSON schema definitions. Plus whatever you find across various git repos.

Not only would you get the ability to build and validate config files for more programs, but there are tons of other types of manifests that can be built.

Some examples of cool new stuff to configure with Nix:

  • Repo git forge (i.e. GitHub, Gitlab, Forgejo, etc.) settings, build pipelines, and most other CI/CD tooling for the repo.
  • Project manifests (i.e.package.json, Cargo.toml, etc.) for virtually every package manager.
  • REST APIs
  • Code / type definitions for tons of programming languages.
  • Kubernetes manifests (using better templating, ergonomics, and validation than with Jinja, Kustomize, or Helm), and tailor them to the nixosConfigurations used to run the cluster nodes.
  • RSS / Atom feeds
  • Editor template snippet definitions
  • DNS records and zones
  • Tree-sitter grammars

Plus tons of services use OpenAPI / Swagger to configure their public REST APIs. The latest spec is compatible with JSON schemas and earlier versions have compatibility vocabs. This would make it super easy to interface with these within Nix.

Not entirely sure of this, but I think I’d have to do most of this at evaluation time, not during the build if I want to use JSON schemas for option declarations, so I’d have to write it in Nix.

Someone correct me if I’m wrong here.

I suppose I could also just create a CLI tool that directly converts JSON schema definitions and spits out Nix module defs, then add those files to nixpkgs, but I’d rather have direct integration with Nix (preferably in nixpkgs).

2 Likes