Sphinx for Python documentation

What is Sphinx?

From sphinx-doc.org:

Sphinx is a tool that makes it easy to create intelligent and beautiful documentation, written by Georg Brandl and licensed under the BSD license.

It was originally created for the new Python documentation, and it has excellent facilities for the documentation of Python projects, but C/C++ is already supported as well, and it is planned to add special support for other languages as well.

Installing Sphinx

You can install Sphinx with pip. If your project uses additional Python packages, you will need those installed in the virtualenv too.

pip install sphinx

Dependencies

Sphinx requires the following packages, so make sure to install them first:

Generating documentation

Create documentation directory

Create a documentation directory within your project directory. This is where the source files for your documentation will be.

mkdir docs
cd docs

Now, you will run a script called sphinx-quickstart to do the setup.

From the Sphinx tutorial:

Sphinx comes with a script called sphinx-quickstart that sets up a source directory and creates a default conf.py with the most useful configuration values from a few questions it asks you.

The script will ask you some setup questions and you can leave the default answer by pressing enter. However, for these questions you will change the answer:

  • when it asks to separate the source and build directories, type y and press enter
  • when it asks for the autodoc extension, type y and press enter

Afterwards, you will notice that your directory structure is something like:

myproject/
|-- README
|-- setup.py
|-- myvirtualenv/
|-- mypackage/
|   |-- __init__.py
|   `-- mymodule.py
`-- docs/
    |-- MakeFile
    |-- build/
    `-- source/

Automatically generating documentation

With autodoc, you can tell Sphinx to look at your docstrings and generate the documentation for your project.

First, open the docs/source/conf.py file to change the configuration. You need to uncomment the following line and add the relative path to where your code is.

sys.path.insert(0, os.path.abspath('../..'))

Now, you will run sphinx-apidoc to automatically generate the .rst files for your module documentation.

sphinx-apidoc -f -o source/ ../mypackage/

The flags used in the previous command are the following:

  • -f — Normally, sphinx-apidoc does not overwrite any files. Use this option to force the overwrite of all files that it generates.

  • -o outputdir — Gives the directory in which to place the generated output.

If you look now in your source directory, you will see new .rst files for each of your modules in addition to the index.rst.

To generate the documentation, from your docs directory run this command:

make html

Using docstrings for generating documentation

Example of docstring used for generating the documentation, from pythonhosted.org.

See more examples.

def public_fn_with_sphinxy_docstring(name, state=None):
    """This function does something.

    :param name: The name to use. 
    :type name: str. 
    :param state: Current state to be in. 
    :type state: bool. 
    :returns: int -- the return code. 
    :raises: AttributeError, KeyError

    """ 
    return 0

Plugins

There are also a couple of IDEs that have plugins that support reStructuredText (ReST) and Sphinx.

PyCharm does not have a Sphinx plugin, but it supports the ReST format for docstrings. Here’s the documentation on how to set that up:

Writing your own documentation

Creating your own ReST (.rst) source files

After using sphinx-quickstart and sphinx-autodoc, you will have a source directory.

myproject/
|-- README
|-- setup.py
|-- myvirtualenv/
|-- mypackage/
`-- docs/
    |-- MakeFile
    |-- build/
    `-- source/         <-- in this directory
        |-- _static/
        |-- _templates/
        |-- index.rst
        `-- (other .rst files auto-generated)

You can create other files here for additional documentation. Once you have created them, then you can include them in the table of contents in index.rst.

Welcome to myproject's documentation!
====================================

.. include:: intro.rst

Contents
--------
.. toctree::
   :maxdepth: 2
   modules

You can include the contents of other .rst files in here with the command .. include:: filename.rst. For the table of contents, you only have to add the filename of the the automatically generated modules.rst.

reStructuredText (ReST) Syntax

If you need a reference to ReST syntax and markup for Sphinx, this is a tutorial that explains it very well.

Including code samples

To include code snippets in your ReST documentation, you could use something like:

.. literalinclude:: path/to/my_script.py
    :language: python
    :lines: 12-13
    :emphasize-lines: 2

Hosting in github pages

If you are using Github for version control and hosting your code, you might be interested in using Github Pages to host and version control your html documentation as well.

Change the build directory

Create new directory

First create a directory for the generated documentation next to the directory for your code. Example:

mkdir myproject-docs

Makefile changes

After setting up the docs directory for your documentation source files, you only need to do a few changes to the configuration to set it up for github pages. This part of the tutorial is based on this example.

Open your myproject/docs/Makefile for editing.

First, change:

BUILDDIR    = build

to:

BUILDDIR    = ../../myproject-docs
PDFBUILDDIR = /tmp
PDF     = ../manual.pdf

The first line points to the new directory we just created. Now, running make html will create an html directory in there.

We will need to make another change in the file. Change these lines:

latexpdf: 
    $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 
    @echo "Running LaTeX files through pdflatex..." 
    make -C $(BUILDDIR)/latex all-pdf 
    @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."

To this:

latexpdf: 
    $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(PDFBUILDDIR)/latex 
    @echo "Running LaTeX files through pdflatex..." 
    make -C $(PDFBUILDDIR)/latex all-pdf 
    cp $(PDFBUILDDIR)/latex/*.pdf $(PDF) 
    @echo "pdflatex finished; see $(PDF)"

Now, the PDF temporary files will be in a tmp directory while it is being built. Afterwards, the resulting PDF will be copied to your main code repo. To generate the files again, run make html.

Create gh-pages branch

First, make sure to commit the code, documentation source files, and the PDF generated from the documentation in your main branch.

Next, change to the directory for your gh-pages repository; in this example, it would be myproject-docs. We will commit this to a new branch gh-pages. We will clone the repository and then create the branch.

cd ../myproject-docs
git clone https://github.com/[user]/[repository].git html
cd html
git branch gh-pages

Now, we need to do some cleanup to remove the files.

git symbolic-ref HEAD refs/heads/gh-pages # auto-switches branches to gh-pages 
rm .git/index 
git clean -fdx

Check if we’re in the gh-pages branch.

git branch

Generate your documentation again with make html. Afterwards, we’re going to add the files and commit them.

git add .
git commit -m "Rebuilt docs"
git push origin gh-pages

Add a .nojekyll file

We need to add an empty file called .nojekyll in our html directory so that Github will render our html instead of looking for jekyll files to render.

touch .nojekyll
git add .nojekyll
git commit -m "Added .nojekyll"
git push origin gh-pages

To view your documentation, go to https://github.com/pages/[user]/[repository]/

Directory structure

You should now have a directory structure similar to this:

myproject/
|-- README
|-- setup.py
|-- myvirtualenv/
|-- mypackage/
|   |-- __init__.py
|   `-- mymodule.py
`-- docs/
    |-- MakeFile
    |-- build/
    `-- source/
myproject-docs/
|-- doctrees/       <-- this directory is autogenerated, but not committed to gh-pages
`-- html/       <-- everything *under* here is committed to the gh-pages branch
    |-- .nojekyll
    |-- index.html
    |-- (more HTML files from sphinx-apidoc reST documents)
    |-- _sources/
    `-- _static/

Further reading

more ...

About Giselle Zeno
I'm a Computer Science PhD student at Purdue and an avid coder. more