- Meeting info
- Past meeting notes
- Documentation office hours in the NixOS Calendar
- Attendees: @hsjobeki @wamirez @kiara @friedow @hgl @lassulus
- Notes by: everyone
vision doc: nix.dev/maintainers/vision.md at master · NixOS/nix.dev · GitHub
Summary
docs.nixos.org is now live as a prototype, but the site is still empty. This will be a live prototype where the community can watch progress and contribute ideas.
We collected the list of sources to ingest (Nixpkgs, NixOS and Nix. Roughly guides & reference material) plus some sidebar.json sidecar for order and hierarchy configuration. (We might need to support multiple formats; i.e. mdBook for nix)
On search there was agreement that search should span all sources and that reusing the existing ElasticSearch backend is worth a proof of concept. There was no agreement on whether search.nixos.org and a search on docs.nixos.org should both exist, or whether one should supersede the other.
After the meeting @hsjobeki and @eveeifyeve developed a rough idea that search.nixos.org should cover larger parts of the ecosystem; while docs.nixos.org only searches in its own content. (guides, options, pkgs, functions, release-notes) both searches seem to have a large technical overlap.
Notes
Which doc sources?
@trespaul asked whether a list of sources exists. @hsjobeki gave roughly the following:
Nixpkgs
- Guides
- Markdown files in the specified format
sidebar.jsonas a sidecar, so we know how to order the guides
- Reference
liband other functions. same data asnoogle.dev/api/v1/data; look into an official JSON pipeline.nixdoccould work, maybe by adding a JSON exportpkgsoptions. fetch from Hydra?- Maybe we can reuse ElasticSearch
- Nixpkgs configuration options (
allowUnfree, etc.)
NixOS
- Guides
sidebar.jsonas a sidecar for ordering
Nix
- Guides (organic prose); Markdown files in the specified format, plus
sidebar.json - Reference
- builtins functions:
nix __dump-language(is this a hack?). Can we get an official JSON export? - CLI commands: JSON?
- Release notes
@hsjobeki: the plan is to add the major sources first, then fill in the gaps.
Search
Options on the table:
- ElasticSearch already exists and could be reused for options and packages. Needs a PoC to see whether result quality is sufficient.
- pagefind for statically rendered pages.
- Hosted alternatives such as Typesense.
@eveeifyeve suggested putting all search into search.nixos.org: one search for everything, and no duplicated effort across search.nixos.org and docs.nixos.org. Concretely: add an option to search.nixos.org that surfaces manual contents in its results.
@trespaul countered that this might go against consolidation, and that the model would bounce users back and forth between search.nixos.org and docs.nixos.org while they are searching for documentation. Since docs.nixos.org needs a search anyway, shouldn’t we consolidate into that one instead? And if all search functionality moves to search.nixos.org, what is left on docs.nixos.org?
@hsjobeki: docs.nixos.org owns the content, so it is natural that the site implements its own search. Under the proposed alternative, would the nixos-search repo have to consume nix.dev and bump inputs after every change to rebuild the index?
@kiara: if we are only reusing the same search engine: our ElasticSearch backend then these are just separate frontends, and it is up to users which one they prefer. Also noted that one page per package/option is nice for link sharing, and that the ElasticSearch configuration currently lives in the Rust files in the search repo.
@friedow: we need to preserve features from search.nixos.org such as filtering results by type (package, option), query strings, and deep linking.
@hsjobeki: we also want a tree search for options.
@trespaul: doing search properly with ElasticSearch depends on figuring out how to integrate with it.
Design and framework
@djacu, relaying from the marketing team: keep using Astro and Tailwind so that a unified design language and shared components can be integrated later.
@hsjobeki noted that @Sigmanificient has previously raised concerns regarding Tailwind. The docs team has not settled on that yet.
Agreement
- Search should search across sources.
- Reuse ElasticSearch, needs a PoC.
Disagreement
- Should both
search.nixos.organd a search ondocs.nixos.orgexist, or should one supersede the other?
Next steps
Investigate:
- ElasticSearch support for full text and function search.
- Whether ElasticSearch can give the UX we want (tree/prefix search, faceted filtering, etc.).
- Whether data in ElasticSearch can be kept in sync with the state of the repos. Currently a CI job runs every 2h and populates ElasticSearch (@kiara).
- How ElasticSearch is configured; see
nixos-search elastic.rs. - Who has access to and control of the ElasticSearch admin and keys, and who has contact with the sponsor (later).
Continue working on:
- The search PoC.
- The Astro site in
github:NixOS/nix.dev/site. - Data pipelines.
- Get content itself ready to be shown (wording & structure, fill gaps)
After the meeting
@hsjobeki reached out to @eveeifyeve. A demo of search may be prepared for the next meeting.
The docs team will release a separate “search” vision that complements the docs vision.
Still to be clarified: does the docs team have authority over search? It seems we lack maintainer rights in the nixos-search repo.