1. Georg Brandl
  2. sphinx


sphinx / HACKING

.. -*- mode: rst -*-

Coding overview

This document tries to give you a cursory overview of the doctools code.

TODO: update this.


The converter doesn't have any dependencies except Python 2.5.

Sphinx needs Python 2.5, Docutils 0.4 (not SVN, because of API changes), Jinja
>= 1.1 (which is at the moment included as an SVN external) and Pygments >= 0.8
(which is optional and can be installed from the cheese shop).

The converter

There's not too much to say about the converter.  It's quite as finished as
possible, and as it has to only work with the body of documentation found in the
Python core, it doesn't have to be as general as possible.

(If other projects using the LaTeX documentation toolchain want to convert their
docs to the new format, the converter will probably have to be amended.)

In ``restwriter.py``, there's some commentary about the inner works of the
converter concerning a single file.

The ``filenamemap.py`` file tells the converter how to rearrange the converted
files in the reST source directories.  There, for example, the tutorial is split
up in several files, and old or unusable files are flagged as not convertable.
Also, non-LaTeX files, such as code include files, are listed to be copied into
corresponding directories.

The directory ``newfiles`` contains a bunch of files that didn't exist in the
old distribution, such as the documentation of Sphinx markup, that will be
copied to the reST directory too.


Sphinx consists of two parts:

* The builder takes the reST sources and converts them into an output format.
  (Presently, HTML, HTML Help or webapp-usable pickles.)

* The web application takes the webapp-usable pickles, which mainly contain the
  HTML bodies converted from reST and some additional information, and turns them
  into a WSGI application, complete with commenting, navigation etc.
  (The subpackage ``web`` is responsible for this.)

An overview of the source files:

  Contains docutils node classes that are not part of standard docutils.  These
  node classes must be handled by every docutils writer that gets one of our

  (The docutils parse a reST document into a tree of "nodes". This nodetree can
  then be converted into an internal representation, XML or anything a Writer
  exists for.)

  Contains the Builder classes, which are responsible for the process of building
  the output files from docutils node trees.

  The builder is called by ``sphinx-build.py``.

  Directive functions that transform our custom directives (like ``.. function::``)
  into doctree nodes.

  The "build environment", a class that holds metadata about all doctrees, and is
  responsible for building them out of reST source files.

  The environment is stored, in a pickled form, in the output directory, in
  order to enable incremental builds if only a few source files change, which
  usually is the case.

  Glue to the Pygments highlighting library.  Will use no highlighting at all if
  that is not installed.  Probably a stripped down version of the Pygments Python
  lexer and HTML formatter could be included.

  HTML help builder helper methods.

_jinja.py, jinja
  The Jinja templating engine, used for all HTML-related builders.

  Helper to keep track of reference count data for the C API reference,
  which is maintained as a separate file.

  Role functions that transform our custom roles (like ``:meth:``) into doctree

  Helper to create a search index for the offline search.

  Directory for all static files for HTML-related builders.

  Directory for Jinja templates, ATM only for HTML.

  General utilities.

  The docutils HTML writer subclass which understands our additional nodes.

Code style

PEP 8 (http://www.python.org/dev/peps/pep-0008) must be observed, with the
following exceptions:

* Line length is limited to 90 characters.
* Relative imports are used, using with the new-in-2.5 'leading dot' syntax.

The file encoding is UTF-8, this should be indicated in the file's first line
with ::

   # -*- coding: utf-8 -*-

Python 3.0 compatibility

As it will be used for Python 3.0 too, the toolset should be kept in a state
where it is fully usable Python 3 code after one run of the ``2to3`` utility.