Splitting the Nixpkgs and NixOS manuals into pages

It’s a well-known problem that the Nixpkgs and NixOS single-page manuals have grown too large for many use cases.

This is why @getpsyched, with some of my guidance, is now building the infrastructure to split it into separate pages – safely. We want URLs to be valid essentially forever, which is why the most important step is to implement a redirects system that will prevent inadvertent breaking changes.

This is building on @pennae’s awesome work porting the manuals from XML to markdown. The new effort is possible thanks to your – the community’s – donations to the NixOS Foundation on Open Collective.

The next steps will be porting the system to 24.05 such that one can navigate from one version to another and always get to the right element. (We already found many broken anchor links while playing around.) After that, we’ll make the split and put the new multi-page-output side by side. The plan is to get the most involved parts done by January 2025.

You can follow progress along the roadmap. Feel free to contact any of us if you have questions or feedback or want to help out.

45 Likes

Cool! Hope we also keep a single-page version, in some situations that’s really useful.

4 Likes

Yes, the single-page version won’t go anywhere, and the redirects are a measure to prevent it from falling apart if contributors happen to stop habitually interacting with it.

4 Likes

This will be brilliant, pleased to hear!

I’m aware of some prior art here, which I’m sure you’ve already seen, but just in case:

@ryantm - Preface | nixpkgs
@pyrox - Preface - Aux Docs

Might be worth comparing notes?

4 Likes

Yeah; we already looked at ryantm’s “spin-off” (for the lack of a better word). Thanks for the ref to Aux docs, will take a look at those too. Will be taking a lot of notes until we get to the splits; along with inspiration from samueldr’s experiment and others.

More suggestions by anyone are very welcome too!

2 Likes

Wow those two options are great.

my vote: I say just double-publish for now to get this and just punt on the URL redirects since we still keep the old model.

2 Likes

Glad to see this work getting done! One of my major gripes with the current docs is the single-page design, which is why I went the direction I did. Loving the new workflow ideas, and maybe your work can help improve mine. I’d love to discuss this more in the Documentation channel in the Nix Matrix space, if you’d like! Would be glad to give insights into how my process works, and why I do a lot of the things I do.

One of the big usability issues with the multi-page design is that trying to find in page wouldn’t work anymore. Do you have a solution for that? Mine is just “rely on material-mkdocs’ search plugin”, but build times are getting longer and longer for it, so I’m definitely looking into how to solve that scalability issue for me.

Feel free to ping in the Documentation room(I’m @pyrox:pyrox.dev there, or if you DM me I can send you my discord username and we can talk there!)

6 Likes

@hsjobeki’s noogle.dev uses pagefind.app for searches.
leaving it here as a reference, in case it could be used.

2 Likes

Now that the enforcement of stable anchors is merged, the next step is adding helper commands that ease dealing with the additional constraints:

Thanks to everyone who already ran into the new workflow for your patience, your feedback helps us getting the kinks ironed out. @katexochen could already make use of the redirects in `buildGoPackage`: remove by katexochen · Pull Request #349478 · NixOS/nixpkgs · GitHub to provide a nice transition for users of the Go build helpers.

10 Likes

Small update on progress: As planned, @GetPsyched transitioned to a new professional endeavor mid December 2024, after completing the redirects mechanism. After some fixups the introduction seems to have went somewhat smoothly and is now reminding contributors to be diligent when removing content, hopefully not annoying everyone too much.

One notable side effect indicating compounding benefits is that we have split out a Nixpkgs release notes section as a consequence of enforcing stable URLs. I still owe a formal introduction of the new section, but it can already be used to pile up important changes that apply to Nixpkgs specifically.

There’s still a bunch of things left to do until we get to the ultimate objective of splitting the Nixpkgs and NixOS manuals into separate pages. If you’re proficient with functional programming, Python, and CSS, and interested in paid work on a highly visible piece of Nix ecosystem documentation (promising both fame and scrutiny!), please contact me here or on Matrix.

13 Likes