There are three different manuals we need to generate, Nix, NixOS and Nixpkgs.
Current situation
Nix
CommonMark files -> mdbook -> HTML
CommonMark files -> ??? -> Man pages
Nixpkgs
CommonMark files -> Pandoc -> Docbook -> HTML
Docbook files -----^
Nix files -> nixdoc ----------^
NixOS
CommonMark files -> Pandoc (at authoring time) -> Docbook files (in repo) -> Docbook -> HTML
??? -> Man pages
Considerations
The reason that NixOS and Nixpkgs are not the same is because the NixOS manual is built whenever we build a new generation of NixOS, so any tools used by the NixOS manual end up in the NixOS build closure. We’d prefer to avoid having Pandoc in there. Ideally, we’d find a tool that has a very small footprint that runs very quickly to avoid bloating NixOS.
Leading contenders for replacing Pandoc/Docbook
Sphinx
nixpkgs manual in CommonMark using sphinx+myst and jupyter-book by FRidh · Pull Request #105036 · NixOS/nixpkgs · GitHub @FRidh made this demo. I don’t know much about it but it is probably fine for nixpkgs and maybe too heavy for NixOS.
https://nix.dev/ also uses this.
mmdoc
GitHub - ryantm/mmdoc: Minimal Markdown Documentation is my work-in-progress tool for making CommonMark to HTML converter. doc: add nixpkgs manual split into multiple pages by ryantm · Pull Request #108063 · NixOS/nixpkgs · GitHub is the PR that integrates that for the nixpkgs manual. Current demo rendering of the nixpkgs manual is here Preface | nixpkgs
Probably suitable for nixpkgs and NixOS, but might not have as many features as people want for the nixpkgs manual and doesn’t have as much resources or development time behind it as the other options.
mdbook
This is a good choice for nixpkgs considering it is already in use for nix. I suspect the Rust toolchain may be too heavy for NixOS, but I haven’t put in the work to verify this.