Why not use YAML for configuration and package declaration?

I’ve been using Salt stack + Jinja templates and I pretty much miss Nix language expressiveness here. For example:

{% set version     = "3.39.0" %}
{% set version_alt = "3/39" %}
{% set url = "https://xxxxx/" + version_alt +
              "/sftpplus-ubuntu1604-x64-" + version + ".tar.gz" %}
# Update hash with `curl -Ls $url | md5sum`
{% set md5 = "3f10fb438bb4ef75dd73e2347c7e34ff" %}

/var/www/SFTPPlus-Server/{{version}}:
  file.directory:
    - user: www-data
    - group: www-data
    - mode: 755
    - makedirs: True
    - require:
      - file: /var/www/SFTPPlus-Server

/var/www/SFTPPlus-Server/{{version}}.tar.gz:
  file.managed:
    - source: {{url}}
    - source_hash: md5={{md5}}
    - user: www-data
    - group: www-data
    - mode: 644

sftpplus:
  cmd.run:
    - name: tar --strip-components=1 -xf ../{{version}}.tar.gz
    - cwd: /var/www/SFTPPlus-Server/{{version}}/
    - runas: www-data
    - user: www-data
    - group: www-data
    - unless: stat /var/www/SFTPPlus-Server/{{version}}/bin
    - require:
      - file: /var/www/SFTPPlus-Server/{{version}}

Here we want to reuse some variables to not duplicate then for various “states”. YAML doesn’t allow such variables, you have to use template language Jinja. While it look simple, it is still a language which you have to learn. Otherwise you’ll have troubles when you try to do something more complicated then variable extract.

For example, you’d like to extend example above and test new version of SFTPPlus. How would you override version args? While it is possible in YAML+Jinja, it is of same level of complexity as Nix solution would be.

Main feature of Nix is that it doesn’t require Jinja template language, it is template language already. Those who create new services, not just use others work, appreciate such a feature.

After Eelco proposal implemented, this will look like:

let
  sftpArgs = <
    sftp.version.major
      | default = "3";
    sftp.version.minor
      | default = "39";
    sftp.version.patch
      | default = "0";
    sftp.version.unpacked_source_md5
      | default = "3f10fb438bb4ef75dd73e2347c7e34ff";
    sftp.version.v1 = with sftp.version; "${major}/${minor}";
    sftp.version.v2 = with sftp.version; "${major}.${minor}.${patch}";
    sftp.sourcePackage = with sftp.version; fetchurl {
      url = "http://xxxx/${v1}/sftpplus-ubuntu1604-x64-${v2}.tar.gz";
      md5 = unpacked_source_md5;
      unpack = True;
    };
  >;

  sftpModule = sftpArgs <
    system.activationScripts.downloadSftp.text = ''
      mkdir -p /var/www/SFTPPlus-Server/
      ln -s ${sftp.sourcePackage} /var/www/SFTPPlus-Server/${sftp.version.v2}
    '';
  >;

  sftpTestingModule = sftpModule <
    sftp.version.major = "3";
    sftp.version.minor = "40";
    sftp.version.patch = "0";
    unpacked_source_md5 = "aaaaaaaaaaaaaaa";
  >;
2 Likes

davidak nixos1@discoursemail.com writes:

(or TOML or anything else common and easy to use?)

I agree with others that nix-lang being a programming language rather
than just a data description language is important.

My own contribution to the bikeshed would be that I find YAML very
opaque and difficult to write or generate. I much prefer JSON or
s-expressions (nix-lang can read and write the former, and the latter is
used by Guix).

4 posts were split to a new topic: Syntax for proposed extensible attrset

It hasn’t been said explicitly but the thing that differentiates nix from other configuration languages is that you can define functions which are fundamental to abstraction.

6 Likes

i don’t think i follow you argument that a language is required as apposed to data
… perhaps this question should have its own topic/thred

any way …
it reminded me of s-expressions (for “symbolic expression”)
#S-expression - Wikipedia

a notation for nested list (tree-structured) data, invented for and popularized by the programming language Lisp, which uses them for source code as well as data.

+1

@tinix If you want to describe complex systems without a lot of repetition, you need a support for abstractions (functions). Using the same representation for data and code (homoiconicity) is orthogonal to this.

2 Likes

It might be helpful to list specific examples of discomfort. Maybe there are ways to improve the language ergonomics without trying a whole new interface.

You might be able to build an interface using TOML along with builtins.fromTOML.

Without nixthe language I feel that nix would struggle with all the issues that other configuration languages suffer from. Take meta.yaml for building conda packages, ansible configuration files, and teraform configuration files. They all have ad-hoc methods for abstraction and you end up with something that is not quite json, yaml, toml, etc. I agree with the steep learning curve and hopefully we can do something to address this.

6 Likes

Maybe there exists a subset of derivations that can be expressed only with pure data. In that case it would be possible to write a mkDataDerivation that takes the data in. Then if there are more advanced use-cases it’s always possible to extend and override.

Not thinking too deeply about this:

name = foo
version = 1.0.0

[source]
type = git
url = https://github.com/foo/bar.git
rev = v1.0.0

[meta]
license = mit
homepage = https://github.com/foo/bar
description = "the traditional bar package"

Already here I want to be able to inject the version in the rev :stuck_out_tongue:

This would be a counter to the natural progression from YAML to YAML+templating. Instead, load the YAML (or TOML) and then extend it programmatically.

3 Likes

I don’t like the idea of loading YAML/TOML and then extending it: it will make things somewhat easier in some simple cases, and considerably more complex in non-trivial cases. Now, instead of learning and understanding Nix, one would need to also learn and understand TOML, how it’s translated into Nix, and how to extend the translations. Not good at all.

1 Like

I’m slowly becoming more and more of a fan of Nix, despite my general dislike of dynamically typed languages and my relative inexperience with functional languages.

I’ve spent years in the trenches with Kubernetes, wading through pools and pools of YAML interspersed with bash and salt and jinja2 templating, and have watched various startups try a myriad of things to improve upon the pain of managing/composing so much “YAML”. I’m pretty solidly convinced that Nix’s approach as a configuration DSL, or Pulumi’s usage of Turing-complete runtimes that can emit config, seem much more reasonable at as soon as any level of abstraction or parameterization is introduced (which there’s plenty of in nixpkgs). Not to be extreme, but trying to imagine nixpkgs as primarily-expressed in yaml is already giving me a headache. In my mind, Nix and YAML are categorically different.

I’d originally liked the idea of having data-driven packages, for some probably large subset of “really normal” and easy-to-build packages, but I think I’d want to see more examples of what it would look like to override (parts of) those package definitions. Since they are just “plain ole objects”, they could still be pure-data Nix files – not as recognizable as JSON/TOML/YAML maybe, but probably still relatively easy to quickly grok, right? Then, when the package inevitably becomes “less” complex, it’s a smaller diff to start adding in the more complicated bits with more Nix syntax.

And since someone else mentioned Terraform, I feel even more conflicted with Terraform and HCL. It’s better than nothing, but I’ve NEVER seen it used on it’s own. Always with a bunch of extra templating/generation/sed-shell-scripts/etc. It also seemed sort of poorly (loosely?) implemented until this most recent “hcl2” rewrite that seems more consistent. I don’t feel similarly constrained or the need to “hack at things” as much when using Nix. Or it feels better supported? Not sure.

As for YAML specifically (as opposed to TOML maybe) it has plenty of its own usability warts. Lots of nearly useless parsers when it comes to error messages, easy to get tripped up on whitespace or semantic oddities around special values that are misinterpreted, etc.

3 Likes

Nix is the worst configuration language ever invented, except for all the others. (Especially YAML.)

If we’re going to support another, simpler configuration language, I think it should be Dhall. It can’t replace Nix entirely, but it can handle lots of NixOS configuration scenarios, just for starters.

8 Likes

Amazing how kindly this post was received. Try going to any other programming community and saying “hey, why do you use this language that sucks for no reason, and not thing that has far fewer features instead?” and see the reactions you get.

15 Likes

What about using JavaScript? :smiley:

2 Likes

Amazing how kindly this post was received. Try going to any other programming community and saying “hey, why do you use this language that sucks for no reason, and not thing that has far fewer features instead?” and see the reactions you get.

And this is why I am happy to be contributing to nixpkgs.

13 Likes

A lot of this can be solved by having the nixos cli tools be able to interact with the system’s configuration file. Right now all that configuration has to be done by directly editing the configuration.nix. The next step for usability is to enable admins to make changed with a cli tool “nixos whatever” and have those changes be reflected in the configuration.nix. A good examble of this would be “nixos install firefox” where firefox gets installed and added to the configuration.nix.

2 Likes

apparently in the context of nix
functions + referential transparency is use full

sandervanderburg/2012/11/an-alternative-explaination-of-nix

There are many causes why function invocations with the same parameters yield different results, such as functions returning time-stamps, generating random numbers, performing I/O, accessing global variables. These causes are called side-effects in the functional programming community.

Because C (and many other commonly used programming languages) allow these side-effects to be programmed, they lack referential transparency , meaning that functions cannot be replaced by its value without changing the behaviour of a program.

1 Like

interesting …
never heard of Dhall before

You can think of Dhall as: JSON + functions + types + imports

it sounds amazing …

after briefly scanning some of the perhaps more complex examples …
dhall-lang/wiki/How-to-translate-recursive-code-to-Dhall

The general algorithm for translating recursive code to non-recursive code is known as Boehm Berarducci encoding and based off of this paper: Automatic synthesis of typed Λ-programs on term algebras

…i feel sufficiently out of my depth
in fact by contrast
makes nix look simple
;]

2 Likes

my pref would be … json

:upside_down_face:

and json makes me think of jq

which apparently has functions and …
‘Most jq builtins are referentially transparent, and yield constant and repeatable value streams when applied to constant inputs. This is not true of I/O builtins.’ - Redirecting to jqlang.github.io

:crazy_face::thinking:

jq wasn’t designed as language from beginning. It is simple when query is simple, but you quickly dive into quirks when your query is out of original jq scope. Nerveless, it’s a nice tool and I made a Nix equivalent some time ago, nixq (nixq - jq but with Nix syntax · GitHub). To be fair, my approach isn’t performant at all (it chokes on large inputs).

4 Likes