A roadmap for the documentation ecosystem

Development Documentation
1

This a wiki post for a roadmap to improve documentation content and infrastructure, spearheaded by @danielsidhion. It can be updated with links to tracking issues, so anyone can follow progress or find something to help out with.

Based on A vision for the manuals (and other documentation), I outlined this roadmap to work towards.

This isn’t a fixed roadmap.
I’ll adapt and change it as needed, but I plan to keep following it at my own pace (I’m a volunteer after all).
It’s also not a complete roadmap, it only has items that I’m particularly interested in working and fits the vision.

— @danielsidhion

Getting to the state outlined in the vision involves a lot of work, and sometimes requires multiple layers of effort to sustainably deliver the vision.
The items below are high-level items and don’t break down or describe in details all the actual work required to complete them.
It’s also possible that some ground-level work can make progress or help multiple of these high-level items.

  • A. Update the content on each manual.
    • A.1. Effort to pay off content debt:
    • A.2. Effort to improve maintainability:
      • Build some sort of tool that points out when content might be out of date.
      • Once this tool exists and is proven to work well, integrate it in the Nixpkgs contribution process to force new documentation to be written as soon as code is changed in a PR.
    • A.3. Effort to reorganise content:
      • Reorganise sections in all manuals.
      • Expose internal Nixpkgs READMEs as an “internal” Nixpkgs manual and move some content from the “external” manual into READMEs.
      • Create a standard for anchor links (currently it’s free form and there all sorts of anchor ids) and build a tool to enforce it.
  • B. Documentation renderers overview I’m currently investigating other documentation frameworks to help with multiple points:
    • Producing manuals in different content formats.
    • Providing multiple ways to navigate content.
    • Provide a unified interface.
    • Point C.1. below.
    • Point D.2. below.
  • C. Improve maintainability of the manuals.
    • C.1. Build or integrate a tool to ensure that every possible link on the manual can still be reached after every change.
    • C.2. Move code from every example in the manuals to their own .nix files and add them to a testing suite.
    • C.3. Document guidelines for documentation writing and enforce a process for reviewing documentation changes, possibly with automatic linting or something similar.
    • C.4. Improve RFC 145 with things we learned from working on it:
      • How to merge doc-strings of functions and their arguments, including attributes of their arguments.
      • How to document default values for arguments and their attributes.
      • How to document attributes/arguments from partially-applied functions.
    • C.5. Unify the toolset used to generate all of our documentation, with ideally as few tools as possible.
  • D. Improve content discoverability.
    • D.1. Expose the manpages for every package built in Nixpkgs on a site, and make the manuals link to it (there might be a GSoD project for this).
    • D.2. Introduce a shared glossary among all manuals and make it easier to link terms to the glossary definitions.
    • D.3. Provide a map of documentation content that people can refer to whenever they feel lost.
15 Likes
2

This can be done in a simple js script for a 404 webpage, where the browser should find the valid path that has the shortest edit distance (re: Levenshtein distance) from the path that was requested from the submitted URL based from the latest sitemap.xml file.

It then presents the suggestion (the shortest edit distance result based off the sitemap.xml links) as a suggested link (i.e. the path determined by the Levenshtein algo) on the 404 page.

example: smart-404-redirect/src/script.ts at 027db02d83e219a7d72b0aa0a22b8aaa870a1d90 · sambacha/smart-404-redirect · GitHub

This can also be easily done right now, by adding pagefind to the build process. It is a static search tool that will index the built HTML pages and provide search capabilities without the overhead of something like typesense/etc.

The manual page is lacking in terms of styling. Syntax highlighting is virtually non-existent, let alone being easily searched / navigate.

Maybe leveraging Pandoc and extending it to provide a ‘Literate Nix’ mode similar to the ‘Literate Haskell’ mode it has would be a possible approach. see https://pandoc.org/MANUAL.html#literate-haskell-support

I think something like the ‘Interactive Graph’ Obsidian has for their docs is what you are alluding to? see https://help.obsidian.md/Home

3

Can someone in the know reflect a bit on the tentative razor(s) for this? (I maintain some documentation in an internal README that I intend to port to the manual when it’s ripe, so I’m hoping to understand whether it would just get ported back out?)

4

The original idea put forward by @infinisil and accepted in the docs team was to separate documentation that is only worth knowing for contributors and maintainers (“inner workings”) from information for mere consumers of Nixpkgs interfaces. IMO that needs to be handled case by case. For example, commit message conventions or documentation of design decisions are only relevant for contributors.

1 Like
5

That item meant something different. I’ll explain with an example: headers in the manual have a specific anchor to them. Look at the link for dockerTools.pullImage. However, as we add/remove/reorganise information in the manual, sometimes it makes sense to change those anchors to something else. But we want to keep the old anchors alive for as long as possible to avoid breaking old links to the manual. Look at this link, which goes to dockerTools.pullImage too. It’s a different anchor - one from an older version of the manual - but which is still valid today.

AFAIK there’s nothing out there which can do this kind of verification for us. Today, we depend 100% on writers and reviewers knowing that we can and should keep old anchors, but I want to automate this.

I’m going to create tracking issues for each one of those items in the roadmap to elaborate and explain what exactly we mean for each item (might take a bit before they’re all there, but the post will be updated whenever the issues are created).

This somewhat fits what I’m researching in item B!

I wasn’t thinking about anything as fancy as that, although I wouldn’t mind if someone came up with something similar! :slight_smile:
I haven’t thought about what I wanted exactly from this yet, but anything that makes it easier for people to see where they should go to find the information they’re looking for is going in the right direction. Some people newer to the ecosystem have said that they have no idea which manual to use, for example. Others aren’t even aware of all the documentation resources we have. I wanted to start by solving this kind of problem.

Thank you for the pointers, btw!

2 Likes
6

About pagefind, if anyone wants to see it on a somewhat on-topic live site, the Jovian NixOS site uses it.

This is an untweaked setup. Stock as it can be, mostly.

2 Likes
7

We already do that in Nix manual. Since we are dealing with fragments, 404 is not enough though:

8

What about just using an entirely new linking methodology instead? Text fragments would side step the issue, Text fragments | MDN

Running the following in devtools gives us this breakdown of the links you provided:

Existing Landscapes

document.querySelectorAll('[id]').forEach((el) => {
  el.style.border = 'solid 2px red';
});

Screenshot 2024-04-01 at 4.24.05 AM
Screenshot 2024-04-01 at 4.24.05 AM2176×2128 484 KB

l

We can see exactly why both those links work, there is nothing else around it! There are no <id>s that could be conflicting to begin with.

Text Fragments

There are Text Fragments which are supported in all browsers except Firefox, they use the following schema for providing granular in-text linking:

Grammar

#:~:text=[prefix-,]start[,end][,-suffix]

There is a proposal to add some additional capabilities to Text fragments

Partition legacy docs into new namespace

Honestly, having looked at some of the ways to get around this, I have to ask: why keep something around that obviously is not working? I am not saying nuke the docs and start fresh, perhaps moving them to a different subdomain and just having the URL requests re-written accordingly could provide a clean enough slate.

Usability, what is it good for?

Consider the existing usability of these links: the pages are not conducive to being consumed by people who may have some impediment to being able to navigate the page, seeing as the pages do not conform to WAI-ARIA landmarks/A11Y, etc. For the greater good, kill it.

All of this to say, without knowing any numbers regarding visits, queries, 404’s, etc, it’s worse than guessing.

usable nix installer paves the way for new users

The nix installer was the biggest stopper in my experience to getting people to use nix, now that is largely solved, people actually are going through the docs and have more questions than answers.

I still can’t get over how useful https://noogle.dev/ is, mashallah there is hope.

Honourable Mentions:

9

Text fragments would have the same issue, and actually make it harder to fix it.
Let me rephrase the issue through an external lens:

Let’s say you write a blog post about something in Nix/Nixpkgs/NixOS, and decide to link to a specific part of the manual in your blog post (which means your URL has a fragment at the end).
With time (this could be tomorrow, in a few weeks, in a year), people will change the manual, reorganise its contents, and if we’re not careful, your link will not work anymore.
The problem we’re solving is: we want those links to still work. That’s why we have an entire section in the Nixpkgs manual that only says “moved to CONTRIBUTING.md”. Those only exist to keep the links alive.

To summarise, this is a problem about the evolution of content in the manual. Backwards compatibility is something we care about.

I hope you can see how text fragments would make this harder to solve: it would force us to “freeze” content so text fragments would still work, thus making it super hard to change the content of the manual.

The way we currently solve this problem works at the human level (i.e. we rely on humans to keep it working). We want to automate it to make it work all the time and avoid human mistakes.

If you kill the current manuals, all that will do is cut access to content from a bunch of people.
The roadmap plans for the effort to provide the content of the manual under a better interface and allow people to consume it through different means. But we’ll strive to be backwards compatible through the migration, and I believe that is 100% possible, just requires a bit of effort in some tooling (which is also accounted for in the roadmap).

4 Likes
10

First of all thanks a lot for your ongoing efforts to improve the documentation!

Sadly, I don’t have the skills to help with technical matters. But I have a proposal for a minor addition to the layout of the manuals that, to me, would improve their readability.

Let me explain: Usually, when I’m familiarizing myself with something new, the official manual is the first place I turn to, and I keep coming back to it as a reference when I have specific questions later. Nix and NixOS are no exception - and I think, overall the manuals for Nix, Nixpkgs and NixOS already do well in this regard. And referring to your point D.3 above, their tables of contents do provide “a map of documentation content that people can refer to whenever they feel lost”. (I assume this isn’t exactly what you meant by “map of documentation”, but still, they provide orientation.)

Sometimes, though, I lose track of where I am when scrolling through the NixOS or Nixpkgs manuals to look something up. In comparison, the Nix Reference Manual with its sidebar, numbered chapters and separate-page-per-node-layout is much more pleasant to read.

I have the feeling that just adding numbers for chapters, sections and subsections would be an effective and simple measure (in the sense of relatively little effort) to improve readability of the other two manuals as well and would make it a lot easier to navigate and find your way through them (even in their current single-page-layout). Something like this:

NixOS Manual
1. Preface
2. Installation
   2.1 Obtaining NixOS
   2.2 Installing NixOS
       2.2.1 Booting from the install medium
       2.2.2 Graphical Installation
       2.2.3 Manual Installation
       2.2.4 Additional installation notes
      ...

(The GNU Guix Reference Manual is actually a good example of this.)

The chapter on “Configuration” of the NixOS manual in particular feels like a jumbled pile that could benefit from chapter and section numbers (and a reordering of the topics).

What do you think, is this worth considering?

4 Likes
11

You’re not the only one!
When you say that you lose track, do you remember the action you were trying to do just before feeling lost, and what goal you were trying to accomplish?

My feeling is that only numbering the chapters/sections might not help without also having a visible list of every chapter/section like the Nix manual does on the sidebar.
However, I asked the question above to try to understand the actions you were taking, because I definitely agree that if adding numbers to chapters/sections is effective, the effort to do so would be relatively small and is something worth look into in the short term until we have a more complete solution.

This is the bare minimum of what we’re trying to provide :slight_smile: The research on that is under item B.

Thank you for the feedback!

2 Likes
12

WARNING: long wall of text incoming! :see_no_evil:

Sections missing clear visual separation

Yes, just recently I tried to optimize my configuration to improve the battery life of my laptop. And I wondered whether tweaking my GPU configuration (Intel Xe) would also help in this regard. That’s when I consulted the NixOS Manual again, specifically its chapter on GPU acceleration. I’m just an average user with no technical background so I’m just vaguely familiar with concepts like “OpenCL”, “Vulkan” etc. So, scrolling through the chapter and just judging by the font sizes of the headings, it was difficult for me to recognize which sections belonged together (AMD/Intel for OpenCL, Vulkan, VA-API) and how they were related to each other (i.e. to distinguish between sections and subsections).

For example: at first glance it isn’t obvious whether Common issues at the end is actually part of the chapter - and if so, whether it’s a subsection to the preceding “VA-API” section or the chapter as a whole. This makes it difficult for a reader to judge at a glance whether it’s relevant for them. Apart from the font size, the chapters, sections and subsections have no visual clues as to what belongs together (they lack clear visual separation). Here, numbering would be very useful for orientation.

Random topics covered

What I also noticed is that the compilation of the topics covered in the manual seems very random. Many of them feel very niche to me (or, which is more likely, I’m just ignorant).

For instance, I wonder for how many users the entry on Castopod is actually relevant and if it justifies inclusion in the authoritative main NixOS Manual (especially now that we have an official wiki). On the other hand, configuration options for battery optimisation on laptops aren’t mentioned at all. In the same vein, the setup of GitLab is explained but GitHub isn’t mentioned; there’s a section for the Pantheon Desktop but not for KDE (and the list goes on).

As I already wrote in my previous post, the whole Configuration chapter gives the impression that whenever anyone found something personally useful in the past, they just slapped it on, without a clear structure in mind.

Restructuring of the content

That’s why, this afternoon, I skimmed through the table of contents (which includes only chapters and sections, but no subsections) and tried to rudimentary sort the various available topics. Due to my limited knowledge, however, I had to lump many sections together again in the “Servers and Services” chapter. Someone more competent than me would have to refine the following provisional proposal - and ideally, sort out many of the very niche topics and maybe add more descriptive headings than just the name of the software. (Many of them I’ve never heard of before and based my sorting just on the first sentence of the description.)

NixOS Manual - (Rearranged)
Version 23.11

Table of Contents

    1. Preface
    2. Installation
        2.1.  Obtaining NixOS
        2.2.  Installing NixOS
        2.3.  Changing the Configuration
        2.4.  Upgrading NixOS
        2.5.  Building a NixOS (Live) ISO
        2.6.  Building Images via systemd-repart
    3. Configuration
        3.1.  System Management
        3.2.  Configuration Syntax
        3.3.  Package Management
        3.4.  User Management
        3.5.  File Systems
        3.6.  Networking
        3.7.  Linux Kernel
        3.8.  External Bootloader Backends
        3.9.  OCS Inventory Agent
        3.10.  Input Methods
        3.11.  Profiles
    4. Desktop Environment
        4.1.  X Window System
        4.2.  Wayland
        4.3.  GNOME Desktop
        4.4.  Xfce Desktop Environment
        4.5.  Pantheon Desktop
    5. Hardware
        5.1.  GPU acceleration
    6. Development
        6.1.  Subversion
        6.2.  GitLab
        6.3.  Sourcehut
        6.4.  Forgejo
        6.5.  Livebook
    7. Applications
        7.1.  Flatpak
        7.2.  Emacs
        7.3.  Oh my ZSH
        7.4.  Digital Bitbox
    8. Backup
        8.1.  BorgBackup
    9. Servers and Services
        9.1.  Matrix
            9.1.1. Mjolnir (Matrix Moderation Tool)
        9.2.  SSL/TLS Certificates with ACME
        9.3.  Garage
        9.4.  Nextcloud
        9.5.  Plausible
        9.6.  Pict-rs
        9.7.  Matomo
        9.8.  Lemmy
        9.9.  Jitsi Meet
        9.10.  Keycloak
        9.11.  Honk
        9.12.  Grocy
        9.13.  GoToSocial
        9.14.  Discourse
        9.15.  c2FmZQ
        9.16.  Akkoma
        9.17.  Meilisearch
        9.18.  Yggdrasil
        9.19.  Prosody
        9.20.  Pleroma
        9.21.  Firefox Sync server
        9.22.  Prometheus exporters
        9.23.  Trezor
        9.24.  parsedmarc
        9.25.  Goss
        9.26.  WeeChat
        9.27.  Taskserver
        9.28.  Apache Kafka
        9.29.  Mailman
        9.30.  Blackfire profiler
        9.31.  Castopod
    10. Databases
        10.1.  Litestream
        10.2.  PostgreSQL
        10.3.  FoundationDB
    11. Containers
        11.1.  Kubernetes
    12. Others
        12.1.  Mosquitto
        12.2.  Cert Spotter
        12.3.  Plotinus
    13. NixOS Administration
        13.1.  Service Management
        13.2.  Rebooting and Shutting Down
        13.3.  User Sessions
        13.4.  Control Groups
        13.5.  Logging
        13.6.  Cleaning the Nix Store
        13.7.  Container Management
        13.8.  Troubleshooting
    14. Nix Development
        14.1.  Getting the Sources
        14.2.  Writing NixOS Modules
        14.3.  Building Specific Parts of NixOS
        14.4.  Experimental feature: Bootspec
        14.5.  What happens during a system switch?
        14.6.  Writing NixOS Documentation
        14.7.  NixOS Tests
        14.8.  Developing the NixOS Test Driver
        14.9.  Testing the Installer
    15. Contributing to this manual
    Appendix
    A. Configuration Options
    B. Release Notes
3 Likes
13

(apologies for the long delay in a reply, I just now managed to clear everything sitting in my inbox)

Thanks for explaining what you were doing when you felt that. I agree with you that numbering in the sections themselves would help without the aid of a navigation sidebar or something like that. I put it on my list to look into whenever I have free time.

That particular section just grabs all existing documentation for modules and surfaces it on the manual. There really is no clear structure to it: the list is figured out after evaluating all modules (this means evaluating Nix code to generate docs!), and is then embedded in the chapter directly.

Reorganising sections in the manuals is part of the roadmap, but I am purposefully not paying much attention to the NixOS manual for now: dealing with the Nixpkgs manual is already a ton of work and takes all the time I can volunteer, but also I’m hoping that a bunch of the content of the NixOS manual will move on to the recently-launched official wiki, which seems like a better place for that sort of information. If this happens, reorganising the NixOS manual at that point will be easier.

3 Likes
14

Thank you for your reply, @danielsidhion ! (And no apologies needed, really. :hugs: )

Very intriguing! I wasn’t aware of that, thanks for pointing it out!

I agree, this sounds very reasonable.

And I am very grateful for the hard work you and everyone else of the Documentation Team is putting into this! Unfortunately, as an average user I’m not knowledgeable and/or confident enough to help, which makes me feel a bit useless. Still, I’m glad to be part of this community - albeit just a consuming part for now. So, again, thank you very much for all you’re doing! :blush: