Writing and building documention for Siconos Software

Documentation overview

The whole Siconos documentation is gathered on Siconos webpage, https://siconos.gforge.inria.fr. It consists in

  • Different “textbooks” : details on algorithms, how to install, use the platform, … everything user and developers should know concerning Siconos software.

    • Getting and installing Siconos software
    • Users’ guide
    • Developers’ guide

    All these guides are generated with sphinx (http://www.sphinx-doc.org/en/master/index.html) from files written in reStructuredText (http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html)

    Source files and sphinx configuration file are all in docs/sphinx directory.

  • Siconos APIs (Python and C++) documentation, automatically generated from inline comments in source files ( *.h/ *.hpp). Rules and best practices to write comments leading to a proper html documentation are detailed in Document source code.

  • In addition to these pages, “inline” documentation for the python API, as usual for python packages, based on docstrings, e.g. :

    >>> import siconos.kernel as sk
    >>> help(sk.SimpleMatrix)
    Help on class SimpleMatrix in module siconos.kernel:
    
    class SimpleMatrix(SiconosMatrix)
    |  Matrix (embedded various types of Boost matrices of double)
    |
    |  SimpleMatrix is used in the platform to store matrices (mathematical object) of
    |  double.
     ...
    

The complete documentation can be generated in one shot using doc target

cd build-dir
cmake -DWITH_DOCUMENTATION=ON path_to_siconos_sources
make doc

It results in html pages, generated in build-dir/docs/build/html. Siconos web site (https://siconos.gforge.inria.fr) is the online version of those pages.

The building process for documentation is described in Building process.

How to write Siconos documentation

Writing textbooks

  • Textbooks are generated from rst files from source-dir/docs/sphinx directory.
  • Everything you need to know to write siconos docs in reStructuredTextPrimer is available here : http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html.
  • All rst files are copied from source dir to binary dir by cmake, so edit only files in source dir.
  • All figures and images must be in sphinx/figures directory.

Document source code

Since source code comments (in c/c++ header files) are used to generate doxygen, sphinx and docstrings documentation, one has to be very careful when writing those comments and follow the rules below.

General rules

files description

Each header file must contain something like

/*! \file SimpleMatrix.hpp
  Brief (no more than one line) description of the content of the file
*/

The name and description will be used in the API contents listings. If this block is not present, the file (and all the objects or functions it contents) won’t appear in the documentation.

Classes and structs

Document each class like this

/** Short description of the class
*
* Detailed description
* equations (see details about latex below), reference to textbooks chapter and so on
*/
class SiconosVector
...

Class methods or functions

/** brief description
 * \param name_of_param1 description of the param
 * \param name_of_param2 description of the param
 * \return description of what is returned
*/
double some_function(int p, int v)

No need to repeat parameters types in comments (param or return)! They will be extracted from function prototype. Something like

/** get size of A
* \param A double A
*  \return unsigned int
*/
unsigned int size(double * A) const;

is totally useless …

rst inside doxygen commments

Use “\rst” / “\endrst” tags to write reStructuredText (reST) specific (i.e. that doxygen can not tackle) comments. See details below for references and math formula.

In the case of comments with leading asterisk, use “\rststar” / “\endrststar” tags

Enums, union …

Since they will probably appear as global variables in python API, it’s important that each component of the enum has an explicit comment, e.g:

/** Global description of the enum */
enum UBLAS_TYPE
{
 /** id for dense matrix or vector */
 DENSE = 1,
 /** id for triangular matrix */
 TRIANGULAR,
}

References to sphinx documents

To refer to any other sphinx document (reminder about sphinx cross-ref : http://www.sphinx-doc.org/en/stable/markup/inline.html) use “\rst” / “\endrst” tags :

/** Class used to defined friction-contact problems

This class deals with blabla

\rst

 See :ref:`global_fc_problem`

\endrst

*/

or with leading asterisk

/** Class used to defined friction-contact problems
 *
 * This class deals with blabla
 *
 * \rststar
 *
 *   See :ref:`global_fc_problem`
 *
 * \endrststar
 *
 *
 */

Math and latex

  • inline math

    use this \f$\alpha\f$ to write inline math
    
  • displayed math

    • Wrap your formula between “rst” and “endrst” tags and write math as you would with sphinx (see http://www.sphinx-doc.org/en/master/ext/math.html).
    • Between rst tags, replace all occurences of :math:’‘dot (one backlash) with :math:’'dot (two backlashes), else doxygen will fail to produce documentation.

    A simple example :

    \rst
    
    .. math::
    
       y &=& h(X,t,\lambda,Z)\\
       R &=& g(X,t,\lambda,Z)
    
    \endrst
    
    • New line after math keyword is required.
    • Indentation for formula (related to math keyword) is required.

    For more complicated maths, use nowrap keyword :

    \rst
    
    .. math::
       :nowrap:
    
        \left\{\begin{array}{l}
        y \geq 0, \lambda \geq 0, y^{T} \lambda=0\\
        if y \leq 0 \quad \mbox{then} \quad \\dot y(t^{+}) - e \\dot y(t^{-}) \geq 0, \quad  \lambda \geq 0, (\\dot y(t^{+}) - e \\dot y(t^{-}))^{T} \lambda=0
        \end{array}\right.
    
    \endrst
    

If you need comments with leading asterisk, use “rststar” / “endrststar” tags :

* \rststar
*
* .. math::
*    :nowrap:
*
*    \begin{eqnarray}
*    \begin{cases}
*     M v =  q +  H r \\
*     u = H^\top v + b \\
*     \hat u = u +\left[
*       \left[\begin{array}{c}
*           \mu^\alpha \|u^\alpha_{T}\|\\
*           0 \\
*           0
*         \end{array}\right]^T, \alpha = 1 \ldots n_c
*      \right]^T \\ \\
*      C^\star_{\mu} \ni {\hat u} \perp r \in C_{\mu}
*     \end{cases}
*    \end{eqnarray}
*
* \endrststar

Building process

One target to generate the whole documentation :

cmake -DWITH_DOCUMENTATION=ON ...
make doc # The whole doc

Resulting files will be in docs/build/html of the current building path.

Below are some details about the documentation generation process, useful only if you want to generate a subpart of the doc or change the configuration and the process.

Tools, config and description

Tools:

  • Doxygen : tool able to generate documentation from annotated C++ sources, in html, xml …
  • Sphinx : powerful generator of documentation (mostly for Python)
  • Breathe : an extension to reStructuredText and Sphinx to be able to read and render the Doxygen xml outputs.
  • Doxy2swig : converter from doxygen XML to SWIG docstring.

Images are sometimes better than words : the different operations are detailed on figures below

Generation of rst files for C++ API

How does it work?

Doxygen is used to generate xml files from comments in headers of each Siconos component. Python scripts are used to postprocess those xml files and produce rst files fitting with Breathe requirements and ready for Sphinx_.

Config and sources:

  • cmake/doxygen_tools.cmake : cmake macros and functions calling doxygen
  • docs/doctools.py : python tools used to generate docs
  • docs/config/doxyxml2sphinx.config.in : doxygen (xml output) for breathe/sphinx
../_images/build_breathe.png

Generation of rst files for C++ API

Generation of rst files for Python API

How does it work?

Doxygen generates xml from comments in headers. Some python scripts are used to postprocess those xml files and produce .i files (swig), ending in docstrings in generated swig python modules. We have written a wrapper to doxy2swig (https://github.com/m7thon/doxy2swig) to fit with our needs. Finally, rst files are generated, based on those docstrings, in autodoc format, for sphinx.

Config and sources:

  • cmake/swig_python_tools.cmake : python functions used to drive docstrings generation
  • docs/doctools.py : python tools used to generate docs
  • docs/sicodoxy2swig.py : python wrapper used to generates docstrings for swig and python. Based on https://github.com/m7thon/doxy2swig.
  • docs/config/doxy2swig.config.in : doxygen (xml output) config, for swig and docstrings
../_images/build_doxy2swig.png

Generation of rst files for Python API

Remark : during generation process, siconos python packages are imported and only objects with non-empty docstrings are documented.

html pages generation

How does it work?

All rst files (from source dir and generated for Python and C++ API) and processed by Sphinx to produce html documentation.

Config and sources:

  • docs/CMakeLists.txt : main driver
  • cmake/doxygen_tools.cmake : cmake macros and functions calling doxygen
  • cmake/doxygen_warning.cmake : included in LibraryProjectSetup, rules to build “*.warnings” files.
  • docs/sphinx/conf.py.in : main sphinx configuration file
  • docs/sphinx/index.rst.in : source for documentation main page
  • docs/sphinx/*/*.rst : inputs for sphinx doc (textbooks)
  • docs/sphinx/figures/* : all figures used in sphinx doc
  • docs/doctools.py : python tools used to generate docs
  • docs/config/doxy.config.in : doxygen (html output) config
  • docs/config/doxy_warnings.config.in : doxygen (log output) config
../_images/build_html_process.png

make doc toolchain

../_images/targets_dep.png

make targets (related to doc) dependencies

Other (exotic) configuration options

  • Full doxygen (i.e. extract all from sources!) html documentation.

    Target : developpers only. Indeed, everything is included in sphinx doc except things like callgraph or collaboration graphs. Use this when those graphs are needed.

    Same as above, with a new cmake option :

cmake -DWITH_DOCUMENTATION=ON -DUSE_DEVEL_DOXYGEN=ON
make doc # The whole doc

Doxygen ouput is set to be “quiet” and without warnings. But, if required (devel), use:

  • Generate doxygen warnings

    cmake -DWITH_DOCUMENTATION=ON -DWITH_DOXYGEN_WARNINGS=ON
    make filter_warnings
    # if WITH_DOXYGEN_WARNINGS_INFILE=ON, create doxygen_warnings/SUMMARY.warnings
    

    It will generate (during compilation process) and print doxygen warnings, either on screen or in files saved in CMAKE_BINARY_DIR/doxygen_warnings (if WITH_DOXYGEN_WARNINGS_INFILE=ON …).

    doxygen warnings conf is defined in docs/config/doxy_warnings.config.in and setup in cmake/doxygen_warnings.cmake.

  • Generate docstrings

    cmake -DWITH_DOCUMENTATION=ON -DWITH_DOXY2SWIG=ON
    make numerics_docstrings
    

    This option is set to ON by default.

    To produce documentation in python interface, xml outputs from doxygen are used to create swig files containing ‘docstrings’ for python.

    Comments written in C++ (doxygen) will be available in python interface, e.g. :

    import siconos.kernel as sk
    help(sk.DynamicalSystem)
    
    Help on class LCP in module siconos.kernel:
    
    class LCP(LinearOSNS)
    |  Non Smooth Problem Formalization and Simulation.
    |
    |  author: SICONOS Development Team - copyright INRIA
    |
    |  This is an abstract class, that provides an interface to define a non smooth
    |  problem:
    |
    |  *   a formulation (ie the way the problem is written)
    |  *   a solver (algorithm and solving formulation, that can be different from
    |      problem formulation)
    |  *   routines to compute the problem solution.
    

Dependencies

  • Doxygen
  • Sphinx, sphinxcontrib.bibtex, sphinxcontrib-youtube, sphinxcontrib-napoleon
  • sphinx-bootstrap-theme
  • Breathe

See docs/requirements.txt for a list of required python packages, and try for example

pip install -r ./docs/requirements.txt
pip install git+https://github.com/sphinx-contrib/youtube.git

See also the file CI/make_siconos_doc.sh that may be helpful to install siconos docs, since it is used by continuous integration on gitlab to provide all dependencies required to build doc on ubuntu.

More about Doxygen to sphinx rst

Some other tools to generate rst from doxygen have been tested : Exhale and doxyrest. We prefer breathe, and notes below are just for the records.

Existing tools (as far as we know …):

Both exhale and doxyrest are available in siconos, (use -DUSE_EXHALE=ON or -DUSE_DOXYREST=ON).

Exhale conf must be defined in conf.py.in (sphinx) and may also handle doxygen run (xml outputs + rst generations from those outputs).

Doxyrest works the same way but is not as convenient as exhale. Outputs are in CMAKE_BINARY_DIR/docs/sphinx/from_doxygen.

Both (exhale and doxyrest) are quite slow and doc generation may take long time …

It seems that it strongly depends on the chosen theme for sphinx (avoid bootswatch).

  • USE_DOXYREST=ON, to generate rst files from xml outputs. Test purpose. Useful to produce rst files from “related pages” of doxygen.
  • USE_EXHALE=ON to generate rst files from xml outputs. Test purpose. Warning : combining this option with EXTRACT_ALL ON (USE_DEVEL_DOXYGEN=ON) may result in a very long time to build documentation.