Happy Docsgiving 🦃

Announcements
#1

Let’s give each other the gift of CommonMark this holiday season! :turkey: :christmas_tree:

Since RFC 72 has been accepted, we can start converting Nixpkgs and NixOS manuals to CommonMark. The sooner we can get the old docs converted, the sooner we can reap the benefits of better output formats and easier contributions!

To get the ball rolling, I’ve started the

CommonMark Docs project

There’s a bunch of issues in the To Do column that you can help with!

There’s also Docbook to CommonMark Best Practices, where we’ll record what we’ve learned as we go along.

How can I help?

  1. Pick an issue in the CommonMark Docs project
  2. Follow the instructions on the issue
  3. …
  4. Profit from unified docs that are easier to contribute to!

Know more about how the docs work?

Please help contribute by setting up more conversion issues and adding them to the project.

32 Likes
#2

Thank you @ryantm for starting to get the ball rolling.

Death to docbook, it’s 382 element types and xml structure :slight_smile:

3 Likes
#3

Really great to see this moving forward.

I probably missed some of the discussion around this, but would you recommend converting each docbook file ‘by hand’, or use a tool like pandoc for an initial pass and then manually fix up the generated markdown (as I’m sure it’s inevitable to do a fair bit of manual work to make it actually right)?

2 Likes
#4

Great timing. I was looking for something small to be my first contribution to nixpkgs :).

7 Likes
#5

Before we start translating individual parts of the documentation: Do we have the tooling in-place to build the hybrid docs for the time being? It would be a pity to have less functional docs as outcome of this. Did the tooling discussion and integration into nixpkgs already happen somewhere?

3 Likes
#6

Exactly, we need to agree on tooling first. This was kept out of the RFC, however the tooling and thereby extensions decides what information gets lost in the conversion. Especially when it comes to references.

1 Like
#7

One problem raised on one of my pull requests is how to handle call outs. There doesn’t seem to be a way to handle them in markdown.

References can be handled by using {#refname} at the end of a heading (only just learnt that tonight thanks jtojnar).

1 Like
#8

Since we already support hybrid Docbook/Markdown sections (at least in the Nixpkgs manual), I don’t think it makes sense to mass convert the entire manual. That sounds too risky and too hard to split up the work.

It might be okay to pass a section through pandoc and then fix it up. I haven’t tested it.

The nixpkgs manual supports hybrid Docbook/Markdown sections. Looks like we’ll need to figure out what to do for NixOS.

No, not yet really. Based on the conversion efforts so far, we’ve run into these issues:

  1. inter-section references
  2. callouts
  3. ShellSession code blocks let you highlight the $/# at the beginning

Yes, we’ll try to do our best. As has been discussed in the RFC, it may be necessary to sacrifice some feature to reach the end goal.

4 Likes
#9

Yay, we had our first successful conversion.

Thank you to wayofthepie and jtojnar.

4 Likes
#10

I already contributed last month the Maven language support section in markdown !

It was pretty easy and I didnt even contemplate more advanced Markdown features.

A style footer that takes you the markdown file though to edit or fix would be A+ though.

I’ve always appreciated that in projects I’ve come to love.

2 Likes
#11

I am having a bit of trouble getting inline docbook tags to work in markdown. pandoc seems to be stripping the tags out of the document.

Does anyone have an example of this working anywhere? I haven’t seen any documents in nixpkgs using it.

1 Like
#12

@wil Here’s an example where an inline-docbook tag worked: https://github.com/NixOS/nixpkgs/blame/2bb9e7c18a6b920074a4fe8bbbfab0e73b066c16/doc/languages-frameworks/beam.section.md#L84

1 Like
#13

If anyone else runs into trouble with docbook tags being stripped like I did there is a work around.

You can escape any docbook code and have pandoc put it directly into the resulting document by putting it inside a code fence with this attribute on it.

```{=docbook}
Put any docbook code here
``
3 Likes
#14

I’ve made a Meta issue for converting from Docbook to CommonMark where we can gather up the best practices.

#15

There’s a new batch of conversion issues in the To Do column of the CommonMark Docs project.

#16

Just a few tips for the people that is contributing.

  1. Use github codespaces if possible to edit the files. It downloads the repo (~1.3GB) way faster than your home internet. Especially if you are from Brazil.

  2. To test locally you can commit in your branch at your fork and push it, you can then download the tarball of the revision that will be way smaller than the whole repo. The url seems like https://github.com/username/nixpkgs/archive/codespaces-1.tar.gz. Untar it and do the nix-build inside the docs folder to test if its nice. You can also install a extension for markdown preview in the codespaces for better feedback loop.

  3. If you are a vim fan, or even a normal user, you can save time replacing common tags to the markdown equivalents like <para> that just go away, <literal> that encloses with backticks and programlisting that is enclosed by backticks. The hard work is to normalize the unnumbered lists, indentation, the special symbols using &something; and wrapping <prompt>s into three backticks (because of the special symbols).

I converted <prompt> blocks into three backticks but without specifying language and converted <programlisting> into the language its written, that is mostly Nix.

1 Like
#17

@ryantm just a little nit that the documentation on nixpkgs itself are a bit different than the issue generated templates.

They mention entering nix-shell before doing nix-build and also running make clean

#18

For those that really want to contribute, many of them can be likely done with a pandoc conversion however please read over the generated markdown to make sure it’s good.

Here is an example pandoc command

nix run nixpkgs.pandoc --command pandoc --from docbook \
          -s doc/builders/packages/weechat.xml \
         --to commonmark \
         -o doc/builders/packages/weechat.section.md
3 Likes
#19

Update: we’ve converted 10 sections so far and 5 more are in progress.

Special thanks to these new (since last update) conversion contributors:
@Mic92, @fzakaria, @AndersonTorres, @wil

3 Likes
#20

Up to 15 conversions done; 9 in progress; and I’ve added more To Do items.

Special thanks to these new (since last update) conversion contributors:
Stupremee, @mmlb, @lucasew

Which sections will you convert?

Visit the CommonMark Docs project and pick a To Do issue today.

1 Like