Why not use YAML for configuration and package declaration?

Development
1

(or TOML or anything else common and easy to use?)

This discussion is related to the Nix roadmap talk from @edolstra at this years NixCon.

I worked as linux administrator and deployed Ansible for configuration management of our servers (250+). It took me just some hours to learn the language and it was easy and even fun to work with!

Nix-lang, Nix and NixOS on the other side have a very steep learning curve. I use NixOS on all my private servers and all desktops (even at work) for some years now and still don’t feel comfortable using it. Some people gave up trying to install NixOS. https://www.reddit.com/r/NixOS/comments/88z7mp/first_impression_of_the_nixos_installation/

While the NixOS configuration feel more like a programming language then a configuration language and nix install firefox don’t work, we will never achieve wider adoption and gain more contributors. It should be the most important point on the roadmap to make using it easier, more obvious and help people solve real world problems in an elegant way.

The goal should be to create the best configuration management, package manager, package definition collection, build tool, linux distribution and what Nix can also solve. We have to look what’s great about Ansible, Docker, brew, debian, … and improve in this direction and also make it easy for users to profit from the things Nix does great. We have to hide all the complicated stuff.

It’s all good to work on technical details that Eelco pointed out in his talk, but i would argue that this is even more important for the project in general and it’s users.

3 Likes
2

Pardon my blunt reaction, but strictly from one user to another, Nix-the-language is a feature, not a bug. NixOS’ configuration being founded on the language is also a feature.

There are plenty of ways to increase appeal to new users without sacrificing what I see to be the core value proposition of the ecosystem. From where I stand, education, documentation, contributions of missing or updated packages, and general advocacy are probably better investments of time and energy than pivoting to a drastically different paradigm that appeals to certain types of users while alienating others.

25 Likes
3

I don’t see how having a steep learning curve with a complicated language nobody else uses is a good feature.
So i ask directly, would it be possible to provide the functionality Nix and NixOS have with a language like YAML and hide all the magic from the user?
I just want to question if it has to be so complicated or if we can improve in this area.

4

No, it’s not possible to have the power of Nix/NixOS with a data language like YAML. It has to be a functional/programming language.

What’s important here is that you can have “magic” implemented in your own configuration. For example, I have a nginx configuration with 15 virtual hosts, each of them defining about 6 services.nginx.virtualHosts.<name> attributes and some non-trivial rewrite rules within. Do I repeat these attributes and rewrite rules for all of 15 virtual hosts? No, I have 15 short specifications for each virtual host, which are translated to aforementioned services.nginx.virtualHosts.<name> attributes and rewrite rules by the code that I have in the configuration. This is only possible because Nix language is a programming language rather than just data in YAML or whatever.

I just want to question if it has to be so complicated or if we can improve in this area.

Well, some things definitely look more complicated/esoteric than than they should be. But probably the best possible improvement would involve creating an accessible and comprehensive knowledge base where people could find various concepts, terms, code examples, explanations, etc…

10 Likes
5

I’ve been using Salt stack + Jinja templates and I pretty much miss Nix language expressiveness here. For example:

{% set version     = "3.39.0" %}
{% set version_alt = "3/39" %}
{% set url = "https://xxxxx/" + version_alt +
              "/sftpplus-ubuntu1604-x64-" + version + ".tar.gz" %}
# Update hash with `curl -Ls $url | md5sum`
{% set md5 = "3f10fb438bb4ef75dd73e2347c7e34ff" %}

/var/www/SFTPPlus-Server/{{version}}:
  file.directory:
    - user: www-data
    - group: www-data
    - mode: 755
    - makedirs: True
    - require:
      - file: /var/www/SFTPPlus-Server

/var/www/SFTPPlus-Server/{{version}}.tar.gz:
  file.managed:
    - source: {{url}}
    - source_hash: md5={{md5}}
    - user: www-data
    - group: www-data
    - mode: 644

sftpplus:
  cmd.run:
    - name: tar --strip-components=1 -xf ../{{version}}.tar.gz
    - cwd: /var/www/SFTPPlus-Server/{{version}}/
    - runas: www-data
    - user: www-data
    - group: www-data
    - unless: stat /var/www/SFTPPlus-Server/{{version}}/bin
    - require:
      - file: /var/www/SFTPPlus-Server/{{version}}

Here we want to reuse some variables to not duplicate then for various “states”. YAML doesn’t allow such variables, you have to use template language Jinja. While it look simple, it is still a language which you have to learn. Otherwise you’ll have troubles when you try to do something more complicated then variable extract.

For example, you’d like to extend example above and test new version of SFTPPlus. How would you override version args? While it is possible in YAML+Jinja, it is of same level of complexity as Nix solution would be.

Main feature of Nix is that it doesn’t require Jinja template language, it is template language already. Those who create new services, not just use others work, appreciate such a feature.

After Eelco proposal implemented, this will look like:

let
  sftpArgs = <
    sftp.version.major
      | default = "3";
    sftp.version.minor
      | default = "39";
    sftp.version.patch
      | default = "0";
    sftp.version.unpacked_source_md5
      | default = "3f10fb438bb4ef75dd73e2347c7e34ff";
    sftp.version.v1 = with sftp.version; "${major}/${minor}";
    sftp.version.v2 = with sftp.version; "${major}.${minor}.${patch}";
    sftp.sourcePackage = with sftp.version; fetchurl {
      url = "http://xxxx/${v1}/sftpplus-ubuntu1604-x64-${v2}.tar.gz";
      md5 = unpacked_source_md5;
      unpack = True;
    };
  >;

  sftpModule = sftpArgs <
    system.activationScripts.downloadSftp.text = ''
      mkdir -p /var/www/SFTPPlus-Server/
      ln -s ${sftp.sourcePackage} /var/www/SFTPPlus-Server/${sftp.version.v2}
    '';
  >;

  sftpTestingModule = sftpModule <
    sftp.version.major = "3";
    sftp.version.minor = "40";
    sftp.version.patch = "0";
    unpacked_source_md5 = "aaaaaaaaaaaaaaa";
  >;
2 Likes
6

davidak nixos1@discoursemail.com writes:

(or TOML or anything else common and easy to use?)

I agree with others that nix-lang being a programming language rather
than just a data description language is important.

My own contribution to the bikeshed would be that I find YAML very
opaque and difficult to write or generate. I much prefer JSON or
s-expressions (nix-lang can read and write the former, and the latter is
used by Guix).

7

4 posts were split to a new topic: Syntax for proposed extensible attrset

11

It hasn’t been said explicitly but the thing that differentiates nix from other configuration languages is that you can define functions which are fundamental to abstraction.

6 Likes
12

i don’t think i follow you argument that a language is required as apposed to data
… perhaps this question should have its own topic/thred

any way …
it reminded me of s-expressions (for “symbolic expression”)
#S-expression - Wikipedia

a notation for nested list (tree-structured) data, invented for and popularized by the programming language Lisp, which uses them for source code as well as data.

+1

13

@tinix If you want to describe complex systems without a lot of repetition, you need a support for abstractions (functions). Using the same representation for data and code (homoiconicity) is orthogonal to this.

2 Likes
14

It might be helpful to list specific examples of discomfort. Maybe there are ways to improve the language ergonomics without trying a whole new interface.

You might be able to build an interface using TOML along with builtins.fromTOML.

15

Without nixthe language I feel that nix would struggle with all the issues that other configuration languages suffer from. Take meta.yaml for building conda packages, ansible configuration files, and teraform configuration files. They all have ad-hoc methods for abstraction and you end up with something that is not quite json, yaml, toml, etc. I agree with the steep learning curve and hopefully we can do something to address this.

6 Likes
16

Maybe there exists a subset of derivations that can be expressed only with pure data. In that case it would be possible to write a mkDataDerivation that takes the data in. Then if there are more advanced use-cases it’s always possible to extend and override.

Not thinking too deeply about this:

name = foo
version = 1.0.0

[source]
type = git
url = https://github.com/foo/bar.git
rev = v1.0.0

[meta]
license = mit
homepage = https://github.com/foo/bar
description = "the traditional bar package"

Already here I want to be able to inject the version in the rev :stuck_out_tongue:

This would be a counter to the natural progression from YAML to YAML+templating. Instead, load the YAML (or TOML) and then extend it programmatically.

3 Likes
17

I don’t like the idea of loading YAML/TOML and then extending it: it will make things somewhat easier in some simple cases, and considerably more complex in non-trivial cases. Now, instead of learning and understanding Nix, one would need to also learn and understand TOML, how it’s translated into Nix, and how to extend the translations. Not good at all.

1 Like
18

I’m slowly becoming more and more of a fan of Nix, despite my general dislike of dynamically typed languages and my relative inexperience with functional languages.

I’ve spent years in the trenches with Kubernetes, wading through pools and pools of YAML interspersed with bash and salt and jinja2 templating, and have watched various startups try a myriad of things to improve upon the pain of managing/composing so much “YAML”. I’m pretty solidly convinced that Nix’s approach as a configuration DSL, or Pulumi’s usage of Turing-complete runtimes that can emit config, seem much more reasonable at as soon as any level of abstraction or parameterization is introduced (which there’s plenty of in nixpkgs). Not to be extreme, but trying to imagine nixpkgs as primarily-expressed in yaml is already giving me a headache. In my mind, Nix and YAML are categorically different.

I’d originally liked the idea of having data-driven packages, for some probably large subset of “really normal” and easy-to-build packages, but I think I’d want to see more examples of what it would look like to override (parts of) those package definitions. Since they are just “plain ole objects”, they could still be pure-data Nix files – not as recognizable as JSON/TOML/YAML maybe, but probably still relatively easy to quickly grok, right? Then, when the package inevitably becomes “less” complex, it’s a smaller diff to start adding in the more complicated bits with more Nix syntax.

And since someone else mentioned Terraform, I feel even more conflicted with Terraform and HCL. It’s better than nothing, but I’ve NEVER seen it used on it’s own. Always with a bunch of extra templating/generation/sed-shell-scripts/etc. It also seemed sort of poorly (loosely?) implemented until this most recent “hcl2” rewrite that seems more consistent. I don’t feel similarly constrained or the need to “hack at things” as much when using Nix. Or it feels better supported? Not sure.

As for YAML specifically (as opposed to TOML maybe) it has plenty of its own usability warts. Lots of nearly useless parsers when it comes to error messages, easy to get tripped up on whitespace or semantic oddities around special values that are misinterpreted, etc.

3 Likes
19

Nix is the worst configuration language ever invented, except for all the others. (Especially YAML.)

If we’re going to support another, simpler configuration language, I think it should be Dhall. It can’t replace Nix entirely, but it can handle lots of NixOS configuration scenarios, just for starters.

8 Likes
20

Amazing how kindly this post was received. Try going to any other programming community and saying “hey, why do you use this language that sucks for no reason, and not thing that has far fewer features instead?” and see the reactions you get.

15 Likes
21

What about using JavaScript? :smiley:

2 Likes
22

Amazing how kindly this post was received. Try going to any other programming community and saying “hey, why do you use this language that sucks for no reason, and not thing that has far fewer features instead?” and see the reactions you get.

And this is why I am happy to be contributing to nixpkgs.

13 Likes
23

A lot of this can be solved by having the nixos cli tools be able to interact with the system’s configuration file. Right now all that configuration has to be done by directly editing the configuration.nix. The next step for usability is to enable admins to make changed with a cli tool “nixos whatever” and have those changes be reflected in the configuration.nix. A good examble of this would be “nixos install firefox” where firefox gets installed and added to the configuration.nix.

2 Likes