Looks good! I see you are already using Antora.
What are the advantages of netlify when we have a general CI like Hydra or Hercules?
Netlify has CI, but the useful part is CD.
The most useful bit for us is live preview for each PR
Once Hercules CI ships impure tasks, youâll be able to deploy Nix artifacts to Netlify with a simple impure glue.
1 Like
My drop in the bikeshed bucket:
I think one of the most important things about documentation tooling is to make it trivial for people to fix documentation issues they come across. Requiring users to learn a new format is non-trivial.
For this reason alone my opinion is that Markdown is the way to go because everyone knows it and almost everyone uses it for documentation. Itâs not about the number of features and ways to generate different highlights if those donât meaningfully improve the quality of the documentation.
Most of the docs issues will always be about how to structure them, how the text in them is phrased etc. and that should outweigh everything else.
Iâve started writing some Nix language documentation in Markdown (nix-1p) and will add more over time as I integrate these into the Nixery documentation.
3 Likes
@tazjin I agree, but thereâs no consensus on that.
As weâre using docbook at the moment, being the mighty god of semantic power, asciidoc seems a good compromise as it at least has good tooling while preserving most of the power.
Saying that Iâm interested to provided good tooling outside Nix/Nixpkgs/NixOS ¡ GitHub ecosystem and hope that one day thereâs going to be recognition that itâs a better path going forward.
1 Like
Markdown is a poor choice for long-form documentation. Itâs good for short snippets but not for e.g. writing a manual.
1 Like
Itâs good for short snippets but not for e.g. writing a manual.
I disagree. For example, most documentation at Google is written in Markdown (with a few minor extensions such as username highlighting) and weâve got thousands of people contributing to it (also see this slide deck).
The format is perfectly fine for 99% of documentation needs, and for the 1% itâs easy enough to âshell outâ to HTML. Whatâs much more important is that the documentation lives directly alongside the code itâs documenting, is easy to find and easy to edit. Docbook is none of those things, and I personally donât see any Asciidoc features that have a significant enough advantage over Markdown to make it worthwhile.
3 Likes
This is a syntax-agnostic benefit.
Markdown is rather limited, youâre forced to âshell outâ to HTML for anything not included in the spec, and itâs got annoying limitations like the inability to write an inline code block that contains a backtick (without shelling out to HTML at least, and writing the following is especially gross: <code>`<code>; it required using ` for the backtick). And if you want to actually generate anything other than HTML, this means you need to not only consume Markdown syntax but also be able to understand HTML syntax too.
Personally, Iâm a fan of reStructuredText. Itâs actually extensible in a standardized fashion and supports more stuff out of the box than Markdown. By default you need two backticks for inline code (e.g. ``foo``), where a single backtick introduces âinterpretedâ text, but we can configure the default role for interpreted text as literal, thus recovering the use of single-backticks. And yes, reStructuredText can encode an inline literal as an inline literal; the double-backtick syntax disables escapes, but interpreted text doesnât, so I can write :literal:`foo\` bar` to get foo ` bar.
2 Likes
This is a syntax-agnostic benefit.
Itâs also cherry-picked from the things I said. The major issues I currently see for Nix documentations are these:
- Existing documentation is outdated
- Existing documentation is hard to find (Google searches often lead me to the XML source of some DocBook page in
nixpkgs, of which I can find the rendered version by quoting an exact string - but this is already too advanced for someone casually browsing) - Not enough people are contributing to documentation (a âdrive-byâ fix in the manual is not an easy feat)
- The structure of the existing documentation is idiosyncratic because itâs neither consistently organised (it grew organically, which is fine up to a point) nor following a principled layout (such as mapping to file paths)
None of these are addressed by whatever syntax we have for writing backticks in inline code literals, but some of them are adversely affected by unfamiliar/niche tooling.
6 Likes
Unfamiliar tooling is a problem, but itâs not as bad as insufficient tooling. If we go with Markdown weâre really asking people to use Markdown plus a specific subset of HTML that we define somewhere with specific naming conventions for things.
Just off the top of my head, three features of our current manuals that Markdown does not support are:
- Callouts, such as the blue âNote:â blocks used everywhere in our manual (example).
- Definition lists, such as used in function documentation.
- Cross-references. Markdown doesnât even have any way of defining an anchor at all except by falling back to HTML. And even if we settle on a specific naming convention with HTML
<a>tags to define anchors, everything breaks the moment we decide to split up the manual into a separate page for each section.
2 Likes
Our Markdown tooling at work uses hoextdown which supports all of those, but without modifying the base language (i.e. all standard Markdown is still valid).
Extensions are sort of inevitable for all of these formats at a certain project size, be it due to an actual need or just for convenience (e.g. we might want to be able to quickly link to the source of an attribute in the package set).
Definition lists, such as used in function documentation
Fun fact #1: When I wrote the tool that generates most of this function documentation I experimented with several other output formats, but decided not to pick the DocBook fight too much at that point. I still think that the format we ended up with isnât the most readable.
Fun fact #2: If you look through some of the generated functions that didnât have its documentation manually edited (such as most things in lib.strings, lib.lists) you will often see that people are actually writing Markdown syntax in the doc comments - itâs deeply engrained in many programmer brains!
3 Likes
What do you think of splitting the discussion between the types of documentation? Otherwise weâll end-up in the same stalemate we had in previous documentation format discussion.
Maybe by splitting the discussion per documentation type, specific requirements might become clearer. We could even use different formats.
Per @domenkozarâs link:
Tutorials
A tutorial:
- is learning-oriented
- allows the newcomer to get started
- is a lesson
Analogy: teaching a small child how to cook
How-to guides
A how-to guide:
- is goal-oriented
- shows how to solve a specific problem
- is a series of steps
Analogy: a recipe in a cookery book
Explanation
An explanation:
- is understanding-oriented
- explains
- provides background and context
Analogy: an article on culinary social history
Reference
A reference guide:
- is information-oriented
- describes the machinery
- is accurate and complete
Analogy: a reference encyclopaedia article
Project documentation
You may well ask: where do things like changelogs, contribution policies, and other information about the project fit into this scheme? The answer is that they do not - because they are, strictly speaking, project documentation rather than documentation of the software itself ,
They can simply be kept in appropriately-named sections alongside the other material - as long as they are not mixed up in it.
8 Likes
@zimbatm: Good point! Agreed.
Tangentially Iâve experimented a bit with generating more documentation out of the nixpkgs tree, first experimentally for NixOS modules.
Check out build-module-docs.nix which makes a âbookâ with a âchapterâ for each NixOS module.
Edit: Check out this example render of NixOS modules. This is very barebones, so consider it just as an idea for now.
6 Likes
I think youâre onto something @tazjin. For technical documentation, the chances of me updating the documentation is much higher if it lives next to the code instead of somewhere in the nixos/doc/ folder. Did you publish the output anywhere to see if Markdown has all the features that we need?
For project documentation specifically we already have some files like the CONTRIBUTING.md file which are forced to be into Markdown due to GitHub. It also doesnât need to be as precise as the reference documentation so it might be acceptable to loose some features. And itâs nice to get a preview for free directly in the PR.
1 Like
GitHub can render reStructuredText files as well (use the extension .rst). Looking at github/markup it looks like it can also handle Textile, RDoc, Org mode, WikiCreole, MediaWiki, AsciiDoc, and perlpod.
2 Likes
@zimbatm Yes, Iâve now published a demo render here.
Itâs quite barebones right now and Iâve done nothing extra to deal with any specific formatting.
NixOS modules were the obvious low-hanging fruit because theyâre in a well structured tree, it makes sense to have per-module documentation and several things follow easily (e.g. pick a doc comment from the top as the module description, pick a README.md in the module folder instead if it exists, âŚ).
This doesnât make as much sense for the package tree (not every package needs a documentation page), but Iâve got some ideas for how to experiment with switching docs in that tree on/off for stuff like the language-specific toolchains.
(Example of a module with a doc comment is my journaldriver module, there are some others around, too).
3 Likes
Regarding shelling out to HTML: That wonât work, because our docs have many targets, including man pages, epubs and pdfs. (brought to my attention by @grahamc on IRC)
while we already have NixOS Search, your effort is encouraging!
The options list is just that - a list of all options. Module documentation should ideally provide some more context around what a module is for and how to use it before listing the options (if you look at pages like the one for journaldriver that I linked youâll get the idea).
One documentation system that does this well is Rustâs (example), where crate/module documentation starts with some descriptive text and examples before the generated reference for the exported code items.
@Infinisil: man pages are an interesting thing I hadnât considered much yet - worth doing some experiments for alternatives to the monolithic man configuration.nix, for example! For other targets, do we know how many people actually actively use those?
1 Like
@tazjin I think I remember at least one person asking for an epub version beforeâŚ! Though Iâm not really sure what youâre supposed to do with that, the NixOS manual is really not meant to be a book or to be read as one. Never heard of anybody wanting the PDF. I feel like those formats might just have been added because theyâre well supported by Docbook.
Iâm using nixpkgs.go-md2man on the direnv project to convert Markdown to man pages. The ROFF format is very limited so I had to edit the Markdown quite a bit until it gives out a nice result.