librelist archives

« back to archive

A successful documentation workflow using Sphinx, doxygen and breathe

A successful documentation workflow using Sphinx, doxygen and breathe

Richard Weickelt
2014-06-19 @ 08:58

I'm new to this list and since librelist does not allow to query the
subcscriber count, I'm quite unsure, how many are reading this, but I guess,
there are quite a couple of people here, who have tried out breathe, hoping
to find an alternative to doxygen and put it aside again because it did not
fit their needs.

I've been trying out Sphinx and breathe lately and I'm pretty sure that we
are just a "breath" away from a maximum flexible C/C++ API documentation
system and I would love to see it happen soon.

I'm interested in how You use breathe and what do You think would it require
in order to use it in a successful way? The following text contains some
thoughts from the view of an C++ library developer.

Doxygen does an excellent job in gathering source code annotations into a
database (XML tree structure). Although it has some limitations when it
comes to heavy template constructs, I found it to behave very robust and
reliable. I don't share the critics, that "Doxygen produces a mindless list
of hyperlinked pages" [1]. The Qt project shows a very good example how to
create a useful API documentation with a doxygen-like generator.

Doxygen's weaknesses reside in its inflexible output rendering engine for
HTML. I spent a lot of time to reduce all the promiscuous navigation
functionality, enhanced the style sheet and ended up with a much cleaner,
but still unsatisfying documentation. These things have bothered me:

1. no template support beside some unreliable CSS hacking and limited
   layout changes
2. no possibility to customize automatically generated overview pages
   (it's in the nature of "automatically generated")
3. no way to show the brief description text before the members list
4. index pages for namespaces do not show brief member description
5. can not mark macros to be related to a namespace, but only a class
6. function signatures always print as tables and show a single parameter
   per line -> consumes a lot of space on the page
7. very limited markdown support, e.g. no definition lists,
   links to figures won't work

I had a look at the source code behind the scene to see how the things work
and if it is possible, to tweak it a little bit. But after all I'm afraid
that it would take ages to understand it.

Now, where can do Sphinx better? I did not try out the details, but for what
I can say so far:
- uses a powerful and very clean markup language (reST)
- includes a good navigation system
- looks "neat" and simple out of the box
- customizable through extensions
- full-text offline search with highlighting of results

Breathe adds
- integration with a powerful documentation system for
  (C++) API documentation
- possibility to create customized overview and index pages
  even for a class list, (namespaces), groups, files...
- possibility to link from every page into the API documentation

Breathe can (until now) not
- link from embedded source code in .rst pages into the API
- generate detailed pages for each class/file

So how would an successful documentation workflow with Sphinx, doxygen and
breathe look for me in the case of a C++ framework or library?
1. Write annotated source code, be expressive and refuse to write
   useless \param and \return blabla where everything can be explained
   in one short sentence. I've already mentioned the Qt documentation,
   that's the holy grail ;-) (e.g. [2]).

2. Create a API documentation database using doxygen. This is doxygen's
   current XML output which can be easily parsed and integrated.

2. Write articles about the big picture, examples, tutorials using Sphinx.

3. Combine everything together in searchable, and clear matter.
   This can be done using breathe. I would then

   - write a reST document for each namespace as namespaces are unlikely
     to change very much over the time,
   - include a directive "doxygennamespace" that can produce a list of
     classes, functions, sub-namespaces. I would have full control, which
     namespace members are shown at which position in the text. This way,
     I can group everything logically. Breathe would then create detailed
     pages for every class that I have included and link to them.
     Details for functions in the namespace could be included with the
     doxygennamespace directive directly on the namespace page and even
     macros which might semantically belong into a namespace.
     Note how this is different to Breath's current functionality, where
     each directive includes the whole content into a single page.
   - do the same for doxygengroups if groups are used instead of namespaces
     if C or another programming language is used.

I've hacked Breathe to generate detailed pages for every class in a given
namespace. This has been done in a global handle very similar to the
autodoxygen handle. It generates a .rst document for each class in a
seperate sub-folder "api" using Breathe's already existing "doxygenclass"
directive. I was wondering, if inter-class links worked and they did.
Now I think, that it would be better, to let the "doxygennamespace"
directive create those pages automatically on-the-fly.

This would result in a win-win:
- use the merits of doxygen and overcome its weaknesses
- gain much more flexibility for the documentation engineer
- write human-readable semantic-driven documentation for comprehensive

Wow, that got much longer than expected. I'm curious about Your thoughts.