How to convert Yaml -> Nix object?

Pegasust · December 5, 2022, 10:03pm

I’m trying to configure program.alacritty in home-manager where program.alacritty.settings takes in an object.

I looked around and there is only builtins.fromJSON. Is there any support for YAML?

I also tried creating a config file “repository” derivation where I put alacritty.yaml through yq to produce alacritty.json. However, it seems pure builds doesn’t allow builtins.fromJSON "${my-derivation}/alacritty/alacritty.json" because it targets absolute path /nix/store/...

Right now, I see some different directions:

Write a library function for fromYAML in Nixlang
Make an escape hatch to use yq to turn YAML → JSON
Correctly create a data-only derivation and read the data contents upon execution
Hack it off and just use something like
xdg.configFile."alacritty/alacritty.yaml" = builtins.readFile "${proj_root}/alacritty/alacritty.yaml"

cdepillabout · December 5, 2022, 10:49pm

I recently had to write a fromYAML in one of my projects, so let me see if I can give you some pointers.

There is no fromYAML builtin in Nix, although there is an open PR adding one:

github.com/NixOS/nix

Add support for parsing YAML

NixOS:master ← NaN-git:yaml

opened 11:35PM - 23 Nov 22 UTC

NaN-git

+48345 -0

# Why `builtins.fromYAML`? YAML is widely used amongst other package managers… and deployment tools. If we want better compatibility to these ecosystems, the ability to parse yaml efficiently in nix is useful, as it is for TOML and JSON, which we already support. Further discussion can be found, i.a., in [#4910](https://github.com/NixOS/nix/issues/4910#issuecomment-1296261500). # Description YAML 1.2 is a complex standard and nix has a limited set of data types. Thus only a subset of YAML can be represented in nix. For example attribute sets require String keys, i.e. attribute sets can represent YAML maps with String keys only, and nix has no data types for binary data or dates. If `builtins.fromYAML` encounters YAML with incompatible data types, then it fails similar to `builtins.fromTOML`. First, the implementation uses [rapidyaml](https://github.com/biojppm/rapidyaml) to parse the YAML string and afterwards the nix objects are created while traversing the YAML tree similar to `builtins.fromTOML`. Tags are mostly ignored and affect only scalars. Custom tags are always ignored and I don't see how custom tags could be handled by nix. # Why rapidyaml? As part of nix robustness and safety of the implementation of `builtins.fromYAML` are important. Some reasons why rapidyaml was chosen: - C++11 library with limited usage of dynamic memory allocations - tested with a wide range of compilers and on different platforms - extensive set of unit tests - heavy usage of assertions within the library - available as single header file, i.e. easy to bootstrap, so that no build tool could link another version of rapidyaml - good speed in comparison to yaml-cpp and other YAML libraries ## limitations of rapidyaml rapidyaml has a few [limitations](https://github.com/biojppm/rapidyaml#known-limitations). Most of these limitations are not really relevant for the nix use case because of the limitations of nix. Some comments with respect to the limitations: - The macro `RYML_WITH_TAB_TOKENS` is defined for `builtins.fromYAML`. - Anchor names terminated with a colon are not fully supported by rapidyaml. `{ &anchor: key: val, anchor: *anchor: }` is parsed correctly, but [this test case](https://github.com/yaml/yaml-test-suite/blob/main/src/2SXE.yaml) fails. The test case is ignored in the `builtins.fromYAML`-tests because it is an edge case and it's no valid YAML 1.3. - Otherwise the only issue that I found is that a string/block containing only tab-, space- and new line-characters might be parsed incorrectly as empty string ([test case](https://github.com/yaml/yaml-test-suite/blob/main/src/Y79Y.yaml)). I don't think that this limits the real world usage. Also rapidyaml parses some invalid YAML successfully, but that is actually helpful. Otherwise the valid JSON `{"a":"b"}`, which is emitted by `builtins.toJSON`, could not be parsed by `builtins.fromYAML` because it is [no valid YAML](https://yaml.org/spec/1.2.2/#separation-spaces) due to the missing separation white space after the `:`-token. # Tested platforms - x86_64-linux - aarch64-linux

If you search on GitHub for fromYAML, you’ll find a bunch of examples of people trying to write this function (although be careful, since some are obviously incorrect).

In my own project, I actually needed a readYAML function (a combination of builtins.readFile and fromYAML), and here is what I came up with:

github.com

cdepillabout/stacklock2nix/blob/65a34bec929e7b0e50fdf4606d933b13b47e2f17/nix/build-support/stacklock2nix/read-yaml.nix


{ runCommand, remarshal }:

# Read a YAML file into a Nix datatype using IFD.
#
# Similar to:
#
# > builtins.fromJSON (builtins.readFile ./somefile)
#
# but takes an input file in YAML instead of JSON.
#
# readYAML :: Path -> a
#
# where `a` is the Nixified version of the input file.
path:

let
  jsonOutputDrv =
    runCommand
      "from-yaml"

This file has been truncated. show original

This internally uses remarshal, but something like yq should definitely work as well.

As long as you control the alacritty.yaml file, one more option for you would be to just write the file in JSON. YAML is a super-set of JSON, so you could just write your alacritty.yaml file in JSON. Then, both alacritty and home-manager could directly read it correctly.