What's Nix's stable API? Is `<CODE>` in `nix-instantiate` part of it?

TL;DR: If you rely on <CODE>, <LAMBDA>, or similar in nix-instantiate --eval’s output syntax, chime in on this PR or in this thread. If you have thoughts on what parts of Nix should be considered a “stable API” and what guarantees a “stable API” should uphold, discuss here. Read on for context.

In service of extending the cannot coerce a set to a string error messages to include the relevant value, I was asked to write a routine to print Nix values in a limited manner (i.e., limiting output to a dozen attributes, to avoid printing huge data structures).

Nix actually has two separate functions for printing values right now: the one used in nix-instantiate --eval output (which prints unevaluated thunks as <CODE> and functions as <LAMBDA>), and the one used in nix repl (which forces values and prints functions as «lambda @ lists.nix:139:5»). The nix repl printer starts with a comment saying it was originally copy-pasted from the nix-instantiate printer, so I figured this was a good time to unify the two printing routines into one.

I chose to base the new printer’s styling on the nix repl printer for a few reasons. First, the nix-instantiate output is ambiguous and not machine-consumable, because <CODE> is valid Nix path syntax:

$ nix-instantiate --eval --expr '{ foo = <CODE>; }'
{ foo = <CODE>; }
$ nix-instantiate --eval --expr '{ foo = {}; }'
{ foo = <CODE>; }

(In fact, @edolstra himself cited this exact reasoning when changing <REPEAT> to «repeated» in nix-instantiate’s output.)

For machine consumption where preserving laziness is important, the nix-instantiate --xml output makes it explicit that the value isn’t a path:

$ nix-instantiate --eval --xml --expr '{ foo = <CODE>; }'
<?xml version='1.0' encoding='utf-8'?>
<expr>
  <attrs>
    <attr column="3" line="1" name="foo">
      <unevaluated />
    </attr>
  </attrs>
</expr>

Second, the nix-instantiate output is inconsistent. Where nix repl’s output makes it clear that text between «chevrons» is “meta-output” and not a literal Nix expression, nix-instantiate uses multiple different conventions: <CODE>, <LAMBDA>, and others use angle brackets, «repeated» and «potential infinite recursion» use chevrons, and there’s also (nullptr) which uses parenthesis.

Finally, nix-instantiate output is uninformative. Where nix repl prints source locations like «lambda @ lists.nix:139:5», nix-instantiate just prints <LAMBDA>. (Here’s a user complaining about this in 2019.)

With all this in mind, I was surprised when my PR was rejected due to changing these parts of the non-strict nix-instantiate output. Surely nobody would consider <CODE> to be for machine consumption? Some members of the Nix team agree — on Matrix, @thufschmitt said that he “certainly wouldn’t recommend” depending on the output, and @tomberek said that “I believe we’ve done plenty of due-diligence for this particular proposal and I am inclined to proceed”.

This issue seems to be partially precipitated by the absence of a written compatibility policy. On Matrix, Nix team members consistently say that Nix is “stable”, but the confines of that stability aren’t defined, so my change was rejected with no rationale more detailed than “I have to be conservative about changes in behavior” (as @roberth said to me in a DM).

However, so far, we haven’t discovered any use case for the <CODE> output that wouldn’t be better-served by using the unambiguous XML output, and we’ve only found one current user of the <CODE> output.

@sternenseemann wants to rely on the <CODE> output for testing laziness. In the Tvix IRC channel, he agreed that “maybe XML is the solution in the end” for this use case, but on the PR itself he requested that the output not change so that the test suite in the Nix repository could be used as an ad-hoc language spec. I’m not very moved by this proposal; Nix’s development shouldn’t be shaped by a rewrite project that’s not yet production-ready, and “checking the same tests with multiple versions of C++ Nix” seems similarly absurd to me — the tests are intrinsically tied to the version of the code they’re committed with, and the tests and code change in lockstep, in response to each other. If the Nix test suite was the stable API, we could never improve Nix’s error messages!

A search for usages of the <CODE> output on GitHub found a project called nix-browse, last updated in 2017, with a README that simply states “Nothing to see here yet…”.

Those are the only users of the <CODE> output in nix-instantiate I’ve been able to find. Do you rely on <CODE> or <LAMBDA> in nix-instantiate --eval output? If so, please chime in with your use case here or on the PR. I don’t want to break anyone’s use case if the alternatives (e.g. XML output) aren’t viable.

4 Likes

Having taken responsibility for Nix documentation, I largely have a user perspective on these matters – take my opinions on implementation with a grain of salt.

That said, @9999years I agree with you on all points, and also lament a lack of clarity around our stability promises, and our lack of focus. I observe that, for my taste, we as maintainers spend too much energy on deliberating over the general direction of contributions rather than assessing and helping improve their implementation quality.

While my comment probably won’t help making progress on the particular issue, I’d sure like to see more effort being put into orienting contributors way before they open pull requests. It’s not like nothing is happening in that regard. But we noticed in the past that, as with tech debt, we also have substantial governance debt that simply takes time to work off. Last month for example, the team discussed how we could get more disciplined with allocating time proportionally to our stated priorities. That alone won’t move the needle much though, since the fundamental bottleneck is maintainer availability, and in my opinion we should get a lot more creative about how the project is run. But there are lots of constraints to our individual and collective agency, so…

Luckily it’s a process anyone can drive with a series of small-scale changes, solid reasoning, and – unfortunately due to the essential and accidental complexity of the project, and very limited maintainer capacity – lots of patience. I warmly invite everyone to make direct, incremental proposals, as this is something we can handle much more easily than substantial changes to the however implicit status quo. Do make many, so you don’t get too attached to any particular one. It’s likely that some will get stuck or lost track of or rejected for all kinds of reasons.

Writing things down one by one, as brilliantly exemplified in the top post, also helps us to make everything more explicit and transparent by being prompted to agree or disagree with a particular statement. One of the biggest problems I see is that most of our domain knowledge is tacit, and there’s too much of it. Exposing that complexity and vagueness is the first step to breaking it down and eventually taming it. That’s what Nix did to package management, that’s why I’m still here.

Contributor patience is a finite resource. When it is exhausted, contributors burn out and leave. I know a half dozen women who have totally abandoned Nix because of this dysfunction. This should not be dismissed so cavalierly when Nix is struggling with maintainer bandwidth!

1 Like

To be clear, I didn’t mean that lightheartedly. Absolutely, everyone’s time and patience is limited, attention is the primary and most scarce resource. What I meant is descriptive: it currently is that way. I don’t like it either, and any non-white-30ish-male contributor leaving out of frustration is nothing less than tragic.

To repeat, I‘m very open to suggestions for changing how things work in order to address all these issues, and assume so are the other maintainers. And while I’m not in the position to decide about such things alone, starting a public discussion is the least we can do.

Follow-up: I reported this as a bug and it got discussed in the Nix team meeting, where @edolstra said “nix-instantiate --eval was never meant to generate a valid Nix expression any way”, but @thufschmitt says “We’ll keep this open for reference, but probably don’t want to act on this”.

Another followup: @edolstra was able to merge a breaking change to actual Nix code functionality, with no discussion and dismissal of the broken use cases. Like <CODE>, it was not documented and had stayed that way for years. Unlike <CODE> there is a documented, relied on use case of this functionality that cannot be done any other way. See Revert "Merge pull request #9902 from NixOS/require-fixed-output-fetchurl" by puckipedia · Pull Request #9911 · NixOS/nix · GitHub.

1 Like