Documentation improvements

[davidak]

Looks good! I see you are already using Antora.

What are the advantages of netlify when we have a general CI like Hydra or Hercules?

[domenkozar]

Netlify has CI, but the useful part is CD.

The most useful bit for us is live preview for each PR

Once Hercules CI ships impure tasks, you’ll be able to deploy Nix artifacts to Netlify with a simple impure glue.

[tazjin]

My drop in the bikeshed bucket:

I think one of the most important things about documentation tooling is to make it trivial for people to fix documentation issues they come across. Requiring users to learn a new format is non-trivial.

For this reason alone my opinion is that Markdown is the way to go because everyone knows it and almost everyone uses it for documentation. It’s not about the number of features and ways to generate different highlights if those don’t meaningfully improve the quality of the documentation.

Most of the docs issues will always be about how to structure them, how the text in them is phrased etc. and that should outweigh everything else.

I’ve started writing some Nix language documentation in Markdown (nix-1p) and will add more over time as I integrate these into the Nixery documentation.

[domenkozar]

@tazjin I agree, but there’s no consensus on that.

As we’re using docbook at the moment, being the mighty god of semantic power, asciidoc seems a good compromise as it at least has good tooling while preserving most of the power.

Saying that I’m interested to provided good tooling outside Nix/Nixpkgs/NixOS · GitHub ecosystem and hope that one day there’s going to be recognition that it’s a better path going forward.

[lilyball]

@tazjin

For this reason alone my opinion is that Markdown is the way to go because everyone knows it and almost everyone uses it for documentation.

Markdown is a poor choice for long-form documentation. It’s good for short snippets but not for e.g. writing a manual.

[tazjin]

It’s good for short snippets but not for e.g. writing a manual.

I disagree. For example, most documentation at Google is written in Markdown (with a few minor extensions such as username highlighting) and we’ve got thousands of people contributing to it (also see this slide deck).

The format is perfectly fine for 99% of documentation needs, and for the 1% it’s easy enough to “shell out” to HTML. What’s much more important is that the documentation lives directly alongside the code it’s documenting, is easy to find and easy to edit. Docbook is none of those things, and I personally don’t see any Asciidoc features that have a significant enough advantage over Markdown to make it worthwhile.

[lilyball]

@tazjin

What’s much more important is that the documentation lives directly alongside the code it’s documenting

This is a syntax-agnostic benefit.

Markdown is rather limited, you’re forced to “shell out” to HTML for anything not included in the spec, and it’s got annoying limitations like the inability to write an inline code block that contains a backtick (without shelling out to HTML at least, and writing the following is especially gross: <code>`<code>; it required using ` for the backtick). And if you want to actually generate anything other than HTML, this means you need to not only consume Markdown syntax but also be able to understand HTML syntax too.

Personally, I’m a fan of reStructuredText. It’s actually extensible in a standardized fashion and supports more stuff out of the box than Markdown. By default you need two backticks for inline code (e.g. ``foo``), where a single backtick introduces “interpreted” text, but we can configure the default role for interpreted text as literal, thus recovering the use of single-backticks. And yes, reStructuredText can encode an inline literal as an inline literal; the double-backtick syntax disables escapes, but interpreted text doesn’t, so I can write :literal:`foo\` bar` to get foo ` bar.

[tazjin]

This is a syntax-agnostic benefit.

It’s also cherry-picked from the things I said. The major issues I currently see for Nix documentations are these:

1. Existing documentation is outdated
2. Existing documentation is hard to find (Google searches often lead me to the XML source of some DocBook page in nixpkgs, of which I can find the rendered version by quoting an exact string - but this is already too advanced for someone casually browsing)
3. Not enough people are contributing to documentation (a “drive-by” fix in the manual is not an easy feat)
4. The structure of the existing documentation is idiosyncratic because it’s neither consistently organised (it grew organically, which is fine up to a point) nor following a principled layout (such as mapping to file paths)

None of these are addressed by whatever syntax we have for writing backticks in inline code literals, but some of them are adversely affected by unfamiliar/niche tooling.

[lilyball]

@tazjin

None of these are addressed by whatever syntax we have for writing backticks in inline code literals, but some of them are adversely affected by unfamiliar/niche tooling.

Unfamiliar tooling is a problem, but it’s not as bad as insufficient tooling. If we go with Markdown we’re really asking people to use Markdown plus a specific subset of HTML that we define somewhere with specific naming conventions for things.

Just off the top of my head, three features of our current manuals that Markdown does not support are:

• Callouts, such as the blue “Note:” blocks used everywhere in our manual (example).
• Definition lists, such as used in function documentation.
• Cross-references. Markdown doesn’t even have any way of defining an anchor at all except by falling back to HTML. And even if we settle on a specific naming convention with HTML <a> tags to define anchors, everything breaks the moment we decide to split up the manual into a separate page for each section.

[tazjin]

Our Markdown tooling at work uses hoextdown which supports all of those, but without modifying the base language (i.e. all standard Markdown is still valid).

Extensions are sort of inevitable for all of these formats at a certain project size, be it due to an actual need or just for convenience (e.g. we might want to be able to quickly link to the source of an attribute in the package set).

Definition lists, such as used in function documentation

Fun fact #1: When I wrote the tool that generates most of this function documentation I experimented with several other output formats, but decided not to pick the DocBook fight too much at that point. I still think that the format we ended up with isn’t the most readable.

Fun fact #2: If you look through some of the generated functions that didn’t have its documentation manually edited (such as most things in lib.strings, lib.lists) you will often see that people are actually writing Markdown syntax in the doc comments - it’s deeply engrained in many programmer brains!

[zimbatm]

What do you think of splitting the discussion between the types of documentation? Otherwise we’ll end-up in the same stalemate we had in previous documentation format discussion.

Maybe by splitting the discussion per documentation type, specific requirements might become clearer. We could even use different formats.

Per @domenkozar’s link:

---

Tutorials

A tutorial:

• is learning-oriented
• allows the newcomer to get started
• is a lesson

Analogy: teaching a small child how to cook

How-to guides

A how-to guide:

• is goal-oriented
• shows how to solve a specific problem
• is a series of steps

Analogy: a recipe in a cookery book

Explanation

An explanation:

• is understanding-oriented
• explains
• provides background and context

Analogy: an article on culinary social history

Reference

A reference guide:

• is information-oriented
• describes the machinery
• is accurate and complete

Analogy: a reference encyclopaedia article

Project documentation

You may well ask: where do things like changelogs, contribution policies, and other information about the project fit into this scheme? The answer is that they do not - because they are, strictly speaking, project documentation rather than documentation of the software itself ,

They can simply be kept in appropriately-named sections alongside the other material - as long as they are not mixed up in it.

[tazjin]

@zimbatm: Good point! Agreed.

Tangentially I’ve experimented a bit with generating more documentation out of the nixpkgs tree, first experimentally for NixOS modules.

Check out build-module-docs.nix which makes a “book” with a “chapter” for each NixOS module.

Edit: Check out this example render of NixOS modules. This is very barebones, so consider it just as an idea for now.

[zimbatm]

I think you’re onto something @tazjin. For technical documentation, the chances of me updating the documentation is much higher if it lives next to the code instead of somewhere in the nixos/doc/ folder. Did you publish the output anywhere to see if Markdown has all the features that we need?

For project documentation specifically we already have some files like the CONTRIBUTING.md file which are forced to be into Markdown due to GitHub. It also doesn’t need to be as precise as the reference documentation so it might be acceptable to loose some features. And it’s nice to get a preview for free directly in the PR.

[lilyball]

@zimbatm

For project documentation specifically we already have some files like the CONTRIBUTING.md file which are forced to be into Markdown due to GitHub.

GitHub can render reStructuredText files as well (use the extension .rst). Looking at github/markup it looks like it can also handle Textile, RDoc, Org mode, WikiCreole, MediaWiki, AsciiDoc, and perlpod.

[tazjin]

@zimbatm Yes, I’ve now published a demo render here.

It’s quite barebones right now and I’ve done nothing extra to deal with any specific formatting.

NixOS modules were the obvious low-hanging fruit because they’re in a well structured tree, it makes sense to have per-module documentation and several things follow easily (e.g. pick a doc comment from the top as the module description, pick a README.md in the module folder instead if it exists, …).

This doesn’t make as much sense for the package tree (not every package needs a documentation page), but I’ve got some ideas for how to experiment with switching docs in that tree on/off for stuff like the language-specific toolchains.

(Example of a module with a doc comment is my journaldriver module, there are some others around, too).

[Infinisil]

Regarding shelling out to HTML: That won’t work, because our docs have many targets, including man pages, epubs and pdfs. (brought to my attention by @grahamc on IRC)

[danbst]

while we already have NixOS Search, your effort is encouraging!

[tazjin]

The options list is just that - a list of all options. Module documentation should ideally provide some more context around what a module is for and how to use it before listing the options (if you look at pages like the one for journaldriver that I linked you’ll get the idea).

One documentation system that does this well is Rust’s (example), where crate/module documentation starts with some descriptive text and examples before the generated reference for the exported code items.

@Infinisil: man pages are an interesting thing I hadn’t considered much yet - worth doing some experiments for alternatives to the monolithic man configuration.nix, for example! For other targets, do we know how many people actually actively use those?

[Infinisil]

@tazjin I think I remember at least one person asking for an epub version before…! Though I’m not really sure what you’re supposed to do with that, the NixOS manual is really not meant to be a book or to be read as one. Never heard of anybody wanting the PDF. I feel like those formats might just have been added because they’re well supported by Docbook.

[zimbatm]

I’m using nixpkgs.go-md2man on the direnv project to convert Markdown to man pages. The ROFF format is very limited so I had to edit the Markdown quite a bit until it gives out a nice result.