Alternative language

[chuwy]

Hey!

I’m quite new to Nix package manager and I really like most of its concepts. However, one thing I keep struggling with is the Nix language, which looks unlike everything else I know. It feels a lot like LISP (and I’m very familiar with functional PLs), but still many things do not click for me, it seems to be full of ad-hoc and unnecessary constructs.

It is very likely that I just don’t have enough experience and eventually I’ll grasp it, but I wanted to ask if there were any initiatives to use something different. Also wondering if there’s anything domain-specific in the Nix language that cannot be replaced with e.g. some LISP dialect + macros.

P.S. This is obviosly a question only to feed my curiousity, I’m not complaining (it is still a great thing!) nor proposing anything (I understand how big this change would be).

Thanks!

[lheckemann]

Yes, there’s Guix which uses the GNU Guile Scheme dialect. AFAIK it still uses the nix daemon, but all the rest is reimplemented in Guile.

[timokau]

Related: Why not use YAML for configuration and package declaration?

[zimbatm]

@chuwy

Also wondering if there’s anything domain-specific in the Nix language that cannot be replaced with e.g. some LISP dialect + macros.

Here are a few language properties that are quite useful to have:

Read-only (no side-effects)

One of the key characteristics of nix is that there are no side-effects to evaluating the language. As long as you have the same inputs you will get the same build recipes. The only place that can be written into is the /nix/store.

This is important to be able to reason about the code and keep it reproducible. I don’t know of other languages that have this builtin as they are mostly designed to interact with the outside world and writing to files it most likely part of the stdlib.

Lazily evaluated

Unlike most other languages, Nix code is only really evaluated when the values are accessed. For example you can write { foo = throw "exception"; bar = 4; }. As long as the “foo” key doesn’t get accessed, the exception won’t be raised.

This can be quite annoying for debugging because it’s much harder to construct stack-traces that makes sense compared to imperative languages. Most people are also unfamiliar with that mental model and tend to read code linearly.

On the other hand, I think this is quite crucial for composing different pieces together. Things like overlays, package overrides and nixos modules all depend on the lazy evaluation to work nicely. This is also important to avoid evaluating all of nixpkgs all the time because it’s so large.

With an imperative language, this can be solved with a level of indirection. Basically add a function call that delays the import of the rest of nixpkgs. This works but all indirection has to be added manually. If one is missing then it means changing the interface.

I don’t know of other languages that have a lazy evaluation model except Haskell.

String context

This is a mostly invisible feature of nix, I don’t think many people know about this.

Strings in nix can have some context attached to them and which is propagated to other strings when they get concatenated together. That context is used to automatically keep track of which derivations a string has been issued from, and then injected into other derivations when they get declared.

For example: pkgs.runCommand "say-hello" {} "echo ${pkgs.cowsay}/bin/cowsay Hello > $out". This is a derivation that outputs a ascii-art cow that says “Hello”. The interesting part is that pkgs.cowsay automatically becomes part of the build of the “say-hello” derivation, just because there is a reference to it in the build recipe. Because the string interpolation casts the pkgs.cowsay to a string, which contains a reference to the pkgs.cowsay derivation in it’s context, which gets propagated to the surrounding string which is added to the derivation produced by the runCommand function. Again, this helps compose derivations together easily. Without needing a ceremony like an interface to declare all the build inputs.

Conclusion

Over time I came to appreciate those properties. I think they are what make the language quite unique and I can’t think of any other existing languages on which those could easily be retro-fitted onto. If anyone has one let me know

[danbst]

yees, string context is quite interesting concept.

Also, lazy+scripty+dynamic typing is fun enough to try it out. I know Lisps can be whatever lazy they want, but I doubt this feature is used (compared to Nix).

@zimbatm

With an imperative language, this can be solved with a level of indirection.

I think this particular case (overlays and overrides) is solved by classes and virtual functions.

[danbst]

Nix package manager isn’t restricted to Nix language. Guix uses Nix under the hood but all the expressions are in Guile.

Nix has an intermediate, compiled expressions called .drv files. It is no longer in Nix language, it is Python language!

In [1]: class Derive: 
   ...:     def __init__(self, *args): self.derivation = args 
   ...:                                                                                                                                         

In [2]: from pprint import pprint                                                                                                               

In [3]: def readDrv(drv): return eval(open(drv).read()).derivation                                                                              

In [4]: pprint(readDrv('/nix/store/gmxsxf5gbipxwhl8s35jzqz2xgvkfpz2-hello-2.10.drv'))                                                           
([('out', '/nix/store/234v87nsmj70i1592h713i6xidfkqyjw-hello-2.10', '', '')],
 [('/nix/store/1p7mib5jjp6kqij706y2j6pjmadhx5ly-stdenv-linux.drv', ['out']),
  ('/nix/store/b3x96w1hy6sxlxk1a1i5fd24b4z6xvv0-hello-2.10.tar.gz.drv',
   ['out']),
  ('/nix/store/ij7yr26mjpnpj78min707x88cbg35sl8-bash-4.4-p23.drv', ['out'])],
 ['/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh'],
 'x86_64-linux',
 '/nix/store/xb062l4b76zyhq6grqf4iyfdikkpg8fl-bash-4.4-p23/bin/bash',
 ['-e', '/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh'],
 [('buildInputs', ''),
  ('builder',
   '/nix/store/xb062l4b76zyhq6grqf4iyfdikkpg8fl-bash-4.4-p23/bin/bash'),
  ('configureFlags', ''),
  ('depsBuildBuild', ''),
  ('depsBuildBuildPropagated', ''),
  ('depsBuildTarget', ''),
  ('depsBuildTargetPropagated', ''),
  ('depsHostHost', ''),
  ('depsHostHostPropagated', ''),
  ('depsTargetTarget', ''),
  ('depsTargetTargetPropagated', ''),
  ('doCheck', '1'),
  ('doInstallCheck', ''),
  ('name', 'hello-2.10'),
  ('nativeBuildInputs', ''),
  ('out', '/nix/store/234v87nsmj70i1592h713i6xidfkqyjw-hello-2.10'),
  ('outputs', 'out'),
  ('patches', ''),
  ('pname', 'hello'),
  ('propagatedBuildInputs', ''),
  ('propagatedNativeBuildInputs', ''),
  ('src', '/nix/store/3x7dwzq014bblazs7kq20p9hyzz0qh8g-hello-2.10.tar.gz'),
  ('stdenv', '/nix/store/cis5rcj9qvm94wq7ch500jnszshc3wx8-stdenv-linux'),
  ('strictDeps', ''),
  ('system', 'x86_64-linux'),
  ('version', '2.10')])

[doronbehar]

I can testify, since I tried Guix before Nix, that the language learning curve was even steeper. However, the main motive for me to try Nix instead wasn’t the language but the community .

[zimbatm]

@danbst

Nix has an intermediate, compiled expressions called .drv files. It is no longer in Nix language, it is Python language!

Nice trick! This is actually a happy coincidence. The .drv files are written in an obscure academic language called “aterm”. It was selected by Eelco because it makes some guarantees in terms of reproducible output like ordering the keys alphabetically.

Nowadays most references to it have disappeared in the nix code. The upstream library has been merged into the nix code base and upstream has disappeared.

[euandreh]

Nice trick! This is actually a happy coincidence. The .drv files are written in an obscure academic language called “aterm”. It was selected by Eelco because it makes some guarantees in terms of reproducible output like ordering the keys alphabetically.

Nowadays most references to it have disappeared in the nix code. The upstream library has been merged into the nix code base and upstream has disappeared.

Wow, I didn’t expect that. This is quite interesting! TIL

Is this the aterm language you mentioned?

http://www.meta-environment.org/doc/books/technology/aterm-guide/aterm-guide.pdf

[zimbatm]

Yep that’s the one. See * Drop the dependency on the ATerm library. · NixOS/nix@f3b8833 · GitHub

[edolstra]

I gave a (rambling) talk on the Nix DSL a couple of weeks ago: https://nixos.org/~eelco/talks/build-systems-nov-2019.pdf

TLDR: the Nix language has all the problems of a general purpose language (namely too much expressiveness, allowing people to build new abstractions that make it harder for other people to understand their code, and making evaluation slow), while not having the features you would expect from a general purpose language, and at the same time not having many domain-specific features (indeed string contexts are one of the few). So it’s not that great as a DSL or a GPL.

[euandreh]

I gave a (rambling) talk on the Nix DSL a couple of weeks ago: https://nixos.org/~eelco/talks/build-systems-nov-2019.pdf

Are there recordings of the symposium? I couldn’t find any reference to
it by searching online.

[edolstra]

No, I don’t think it was recorded.

[SRGOM]

I’ve been off computers for a while but if there were an inherit (x) a b equivalent for arrays, and with were eliminated, nix as a language would be extremely small and reasonably consistent and easy to read. I would blame with keyword for most of nix’s problems.

[pblkt]

@edolstra do you mind rehashing here the part in that talk about the alternatives - slide 17? (also for posterity’s sake )

I think it’s useful to have an explicit discussion about what is it that the alternatives cannot express that’s required for the Nix use case.

[jonringer]

@pblkt

@edolstra do you mind rehashing here the part in that talk about the alternatives - slide 17? (also for posterity’s sake )

what do you mean rehashing? He expanded on his sentiment in another post

@edolstra

TLDR: the Nix language has all the problems of a general purpose language (namely too much expressiveness, allowing people to build new abstractions that make it harder for other people to understand their code, and making evaluation slow), while not having the features you would expect from a general purpose language, and at the same time not having many domain-specific features (indeed string contexts are one of the few). So it’s not that great as a DSL or a GPL.

[pblkt]

…which is the post that I replied to

I’m assuming that more was said in the talk covering the alternatives than what’s in the slide, and was hoping to have it written here in context of the alternatives discussion

Especially interesting are the cases that are core to the Nix use case but other configuration languages are not expressive enough to cover

[jonringer]

There’s a good podcast at https://www.youtube.com/watch?v=JKJTfrGNMPY covering the topic of configuration languages. There’s a lot of similarities between dhall and nix-lang.

I actually quite like Nix (lang), I think it’s really specialized for doing package configuration, but it fits the role really well without few oddities.

[aszenz]

As a beginner to nix language, to me the most frustrating aspect of it has been the complete lack of tooling around nix, unlike other real languages there is no autocomplete, no documentation on hover, nothing that tells me what arguments this function takes, no goto definition and so on.

I’m not sure if this lack of tooling is related to the way the language is designed internally that makes it harder for such tools to exist.

I did watch a recent nix con talk on improving nix ergonomics with modules and I’m optimistic about that.

An alternative language that inherits this lack of tooling and documentation while merely providing a more common syntax won’t provide much benefit I suppose.

[jonringer]

these things are harder to implement with functional programming languages. OOP languages have the benefit of having all the classes, methods, fields, and properties known ahead of time.

In functional programming you’ll say something like:

  valid_values = filter (x: x.is_valid) all_values;

It’s hard to know what you were meaning to type. Some languages have “type holes”, but generally you still need to write the majority of the expression, and leave only one “hole” open. Unfortunately, nix doesn’t have this as it’s dynamically typed.

But in general, I agree, I had to “cut my teeth” by looking at other code within nixpkgs. And the IDE experience isn’t great. I just use vim + coc + vim-nix. But, I generally have to know ahead of time, the code I need to write.

In defense of functional programming, it takes me a bit longer to get started because I have to learn the “prelude” of functions that are commonly used. The most common functions (map, fold, filter, etc) usually carry over between languages, so if you’re familiar with another functional programming language, then learning this is pretty quick.

(rant warning)
Another benefit of functional programming is that I don’t usually experience, “I need to do X, and class Y looks like the right class. Let me learn how this one class is defined, and it’s very similar to this other class Z; but since it needs to differ in this one regard, it should be it’s own class. Oh, and since class Z was sealed, this makes sense why they didn’t make class Y a subclass of Z; but now it uses slightly different conventions so it’s similar but not the same now. Oh, and I forgot that this class’s parent-parent-parent-class already implements this, and I should have just done that.”