David Malcolm avatar David Malcolm committed 1e46012

(dmalcolm, lac): Mass rename of documentation files from .txt to .rst, to help editors recognize the format

Script used (in pypy/doc):
for f in $(find -name "*.txt"); do hg rename $f $(echo $f | sed -e"s|.txt|.rst|"); done

Comments (0)

Files changed (568)

pypy/doc/__pypy__-module.rst

+=======================
+The ``__pypy__`` module
+=======================
+
+The ``__pypy__`` module is the main entry point to special features provided
+by PyPy's standard interpreter. Its content depends on `configuration options`_ 
+which may add new functionality and functions whose existence or non-existence 
+indicates the presence of such features. 
+
+.. _`configuration options`: config/index.html
+
+Generally available functionality
+=================================
+
+ - ``internal_repr(obj)``: return the interpreter-level representation of an
+   object.
+ - ``bytebuffer(length)``: return a new read-write buffer of the given length.
+   It works like a simplified array of characters (actually, depending on the
+   configuration the ``array`` module internally uses this).
+
+Thunk Object Space Functionality
+================================
+
+When the thunk object space is used (choose with :config:`objspace.name`),
+the following functions are put into ``__pypy__``:
+
+ - ``thunk``
+ - ``is_thunk``
+ - ``become``
+ - ``lazy``
+
+Those are all described in the `interface section of the thunk object space
+docs`_.
+
+For explanations and examples see the `thunk object space docs`_.
+
+.. _`thunk object space docs`: objspace-proxies.html#thunk
+.. _`interface section of the thunk object space docs`: objspace-proxies.html#thunk-interface
+
+Taint Object Space Functionality
+================================
+
+When the taint object space is used (choose with :config:`objspace.name`),
+the following names are put into ``__pypy__``:
+
+ - ``taint``
+ - ``is_tainted``
+ - ``untaint``
+ - ``taint_atomic``
+ - ``_taint_debug``
+ - ``_taint_look``
+ - ``TaintError``
+
+Those are all described in the `interface section of the taint object space
+docs`_.
+
+For more detailed explanations and examples see the `taint object space docs`_.
+
+.. _`taint object space docs`: objspace-proxies.html#taint
+.. _`interface section of the taint object space docs`: objspace-proxies.html#taint-interface
+
+Transparent Proxy Functionality
+===============================
+
+If `transparent proxies`_ are enabled (with :config:`objspace.std.withtproxy`)
+the following functions are put into ``__pypy__``:
+
+ - ``tproxy(typ, controller)``: Return something that looks like it is of type
+   typ. Its behaviour is completely controlled by the controller. See the docs
+   about `transparent proxies`_ for detail.
+
+ - ``get_tproxy_controller(obj)``: If obj is really a transparent proxy, return
+   its controller. Otherwise return None.
+
+.. _`transparent proxies`: objspace-proxies.html#tproxy
+
+
+Functionality available on py.py (not after translation)
+========================================================
+
+ - ``isfake(obj)``: returns True if ``obj`` is faked.
+
+ - ``interp_pdb()``: start a pdb at interpreter-level.
+
+
+

pypy/doc/__pypy__-module.txt

-=======================
-The ``__pypy__`` module
-=======================
-
-The ``__pypy__`` module is the main entry point to special features provided
-by PyPy's standard interpreter. Its content depends on `configuration options`_ 
-which may add new functionality and functions whose existence or non-existence 
-indicates the presence of such features. 
-
-.. _`configuration options`: config/index.html
-
-Generally available functionality
-=================================
-
- - ``internal_repr(obj)``: return the interpreter-level representation of an
-   object.
- - ``bytebuffer(length)``: return a new read-write buffer of the given length.
-   It works like a simplified array of characters (actually, depending on the
-   configuration the ``array`` module internally uses this).
-
-Thunk Object Space Functionality
-================================
-
-When the thunk object space is used (choose with :config:`objspace.name`),
-the following functions are put into ``__pypy__``:
-
- - ``thunk``
- - ``is_thunk``
- - ``become``
- - ``lazy``
-
-Those are all described in the `interface section of the thunk object space
-docs`_.
-
-For explanations and examples see the `thunk object space docs`_.
-
-.. _`thunk object space docs`: objspace-proxies.html#thunk
-.. _`interface section of the thunk object space docs`: objspace-proxies.html#thunk-interface
-
-Taint Object Space Functionality
-================================
-
-When the taint object space is used (choose with :config:`objspace.name`),
-the following names are put into ``__pypy__``:
-
- - ``taint``
- - ``is_tainted``
- - ``untaint``
- - ``taint_atomic``
- - ``_taint_debug``
- - ``_taint_look``
- - ``TaintError``
-
-Those are all described in the `interface section of the taint object space
-docs`_.
-
-For more detailed explanations and examples see the `taint object space docs`_.
-
-.. _`taint object space docs`: objspace-proxies.html#taint
-.. _`interface section of the taint object space docs`: objspace-proxies.html#taint-interface
-
-Transparent Proxy Functionality
-===============================
-
-If `transparent proxies`_ are enabled (with :config:`objspace.std.withtproxy`)
-the following functions are put into ``__pypy__``:
-
- - ``tproxy(typ, controller)``: Return something that looks like it is of type
-   typ. Its behaviour is completely controlled by the controller. See the docs
-   about `transparent proxies`_ for detail.
-
- - ``get_tproxy_controller(obj)``: If obj is really a transparent proxy, return
-   its controller. Otherwise return None.
-
-.. _`transparent proxies`: objspace-proxies.html#tproxy
-
-
-Functionality available on py.py (not after translation)
-========================================================
-
- - ``isfake(obj)``: returns True if ``obj`` is faked.
-
- - ``interp_pdb()``: start a pdb at interpreter-level.
-
-
-

pypy/doc/_ref.rst

+.. _`demo/`: ../../demo
+.. _`demo/pickle_coroutine.py`: ../../demo/pickle_coroutine.py
+.. _`lib-python/`: ../../lib-python
+.. _`lib-python/2.5.2/dis.py`: ../../lib-python/2.5.2/dis.py
+.. _`annotation/`:
+.. _`pypy/annotation`: ../../pypy/annotation
+.. _`pypy/annotation/annrpython.py`: ../../pypy/annotation/annrpython.py
+.. _`annotation/binaryop.py`: ../../pypy/annotation/binaryop.py
+.. _`pypy/annotation/builtin.py`: ../../pypy/annotation/builtin.py
+.. _`pypy/annotation/model.py`: ../../pypy/annotation/model.py
+.. _`bin/`: ../../pypy/bin
+.. _`config/`: ../../pypy/config
+.. _`pypy/config/pypyoption.py`: ../../pypy/config/pypyoption.py
+.. _`doc/`: ../../pypy/doc
+.. _`doc/config/`: ../../pypy/doc/config
+.. _`doc/discussion/`: ../../pypy/doc/discussion
+.. _`interpreter/`:
+.. _`pypy/interpreter`: ../../pypy/interpreter
+.. _`pypy/interpreter/argument.py`: ../../pypy/interpreter/argument.py
+.. _`interpreter/astcompiler/`:
+.. _`pypy/interpreter/astcompiler`: ../../pypy/interpreter/astcompiler
+.. _`pypy/interpreter/executioncontext.py`: ../../pypy/interpreter/executioncontext.py
+.. _`pypy/interpreter/function.py`: ../../pypy/interpreter/function.py
+.. _`interpreter/gateway.py`:
+.. _`pypy/interpreter/gateway.py`: ../../pypy/interpreter/gateway.py
+.. _`pypy/interpreter/generator.py`: ../../pypy/interpreter/generator.py
+.. _`pypy/interpreter/mixedmodule.py`: ../../pypy/interpreter/mixedmodule.py
+.. _`pypy/interpreter/module.py`: ../../pypy/interpreter/module.py
+.. _`pypy/interpreter/nestedscope.py`: ../../pypy/interpreter/nestedscope.py
+.. _`pypy/interpreter/pyopcode.py`: ../../pypy/interpreter/pyopcode.py
+.. _`interpreter/pyparser/`:
+.. _`pypy/interpreter/pyparser`: ../../pypy/interpreter/pyparser
+.. _`pypy/interpreter/pyparser/pytokenizer.py`: ../../pypy/interpreter/pyparser/pytokenizer.py
+.. _`pypy/interpreter/pyparser/parser.py`: ../../pypy/interpreter/pyparser/parser.py
+.. _`pypy/interpreter/pyparser/pyparse.py`: ../../pypy/interpreter/pyparser/pyparse.py
+.. _`pypy/interpreter/pyparser/future.py`: ../../pypy/interpreter/pyparser/future.py
+.. _`pypy/interpreter/pyparser/metaparser.py`: ../../pypy/interpreter/pyparser/metaparser.py
+.. _`pypy/interpreter/astcompiler/astbuilder.py`: ../../pypy/interpreter/astcompiler/astbuilder.py
+.. _`pypy/interpreter/astcompiler/optimize.py`: ../../pypy/interpreter/astcompiler/optimize.py
+.. _`pypy/interpreter/astcompiler/codegen.py`: ../../pypy/interpreter/astcompiler/codegen.py
+.. _`pypy/interpreter/astcompiler/tools/asdl_py.py`: ../../pypy/interpreter/astcompiler/tools/asdl_py.py
+.. _`pypy/interpreter/astcompiler/tools/Python.asdl`: ../../pypy/interpreter/astcompiler/tools/Python.asdl
+.. _`pypy/interpreter/astcompiler/assemble.py`: ../../pypy/interpreter/astcompiler/assemble.py
+.. _`pypy/interpreter/astcompiler/symtable.py`: ../../pypy/interpreter/astcompiler/symtable.py
+.. _`pypy/interpreter/astcompiler/asthelpers.py`: ../../pypy/interpreter/astcompiler/asthelpers.py
+.. _`pypy/interpreter/astcompiler/ast.py`: ../../pypy/interpreter/astcompiler/ast.py
+.. _`pypy/interpreter/typedef.py`: ../../pypy/interpreter/typedef.py
+.. _`lib/`:
+.. _`lib_pypy/`: ../../lib_pypy
+.. _`lib/distributed/`: ../../lib_pypy/distributed
+.. _`lib_pypy/stackless.py`: ../../lib_pypy/stackless.py
+.. _`lib_pypy/pypy_test/`: ../../lib_pypy/pypy_test
+.. _`module/`:
+.. _`pypy/module`:
+.. _`pypy/module/`: ../../pypy/module
+.. _`pypy/module/__builtin__/__init__.py`: ../../pypy/module/__builtin__/__init__.py
+.. _`pypy/module/_stackless/test/test_clonable.py`: ../../pypy/module/_stackless/test/test_clonable.py
+.. _`pypy/module/_stackless/test/test_composable_coroutine.py`: ../../pypy/module/_stackless/test/test_composable_coroutine.py
+.. _`objspace/`:
+.. _`pypy/objspace`: ../../pypy/objspace
+.. _`objspace/dump.py`: ../../pypy/objspace/dump.py
+.. _`objspace/flow/`: ../../pypy/objspace/flow
+.. _`objspace/std/`:
+.. _`pypy/objspace/std`: ../../pypy/objspace/std
+.. _`objspace/taint.py`: ../../pypy/objspace/taint.py
+.. _`objspace/thunk.py`:
+.. _`pypy/objspace/thunk.py`: ../../pypy/objspace/thunk.py
+.. _`objspace/trace.py`:
+.. _`pypy/objspace/trace.py`: ../../pypy/objspace/trace.py
+.. _`pypy/rlib`:
+.. _`rlib/`: ../../pypy/rlib
+.. _`pypy/rlib/rarithmetic.py`: ../../pypy/rlib/rarithmetic.py
+.. _`pypy/rlib/test`: ../../pypy/rlib/test
+.. _`pypy/rpython`:
+.. _`pypy/rpython/`:
+.. _`rpython/`: ../../pypy/rpython
+.. _`rpython/lltypesystem/`: ../../pypy/rpython/lltypesystem
+.. _`pypy/rpython/lltypesystem/lltype.py`:
+.. _`rpython/lltypesystem/lltype.py`: ../../pypy/rpython/lltypesystem/lltype.py
+.. _`rpython/memory/`: ../../pypy/rpython/memory
+.. _`rpython/memory/gc/generation.py`: ../../pypy/rpython/memory/gc/generation.py
+.. _`rpython/memory/gc/hybrid.py`: ../../pypy/rpython/memory/gc/hybrid.py
+.. _`rpython/memory/gc/markcompact.py`: ../../pypy/rpython/memory/gc/markcompact.py
+.. _`rpython/memory/gc/marksweep.py`: ../../pypy/rpython/memory/gc/marksweep.py
+.. _`rpython/memory/gc/semispace.py`: ../../pypy/rpython/memory/gc/semispace.py
+.. _`rpython/ootypesystem/`: ../../pypy/rpython/ootypesystem
+.. _`rpython/ootypesystem/ootype.py`: ../../pypy/rpython/ootypesystem/ootype.py
+.. _`rpython/rint.py`: ../../pypy/rpython/rint.py
+.. _`rpython/rlist.py`: ../../pypy/rpython/rlist.py
+.. _`rpython/rmodel.py`: ../../pypy/rpython/rmodel.py
+.. _`pypy/rpython/rtyper.py`: ../../pypy/rpython/rtyper.py
+.. _`pypy/rpython/test/test_llinterp.py`: ../../pypy/rpython/test/test_llinterp.py
+.. _`pypy/test_all.py`: ../../pypy/test_all.py
+.. _`tool/`: ../../pypy/tool
+.. _`tool/algo/`: ../../pypy/tool/algo
+.. _`tool/pytest/`: ../../pypy/tool/pytest
+.. _`pypy/translator`:
+.. _`translator/`: ../../pypy/translator
+.. _`translator/backendopt/`: ../../pypy/translator/backendopt
+.. _`translator/c/`: ../../pypy/translator/c
+.. _`translator/cli/`: ../../pypy/translator/cli
+.. _`translator/goal/`: ../../pypy/translator/goal
+.. _`pypy/translator/goal/targetnopstandalone.py`: ../../pypy/translator/goal/targetnopstandalone.py
+.. _`translator/jvm/`: ../../pypy/translator/jvm
+.. _`translator/stackless/`: ../../pypy/translator/stackless
+.. _`translator/tool/`: ../../pypy/translator/tool
+.. _`translator/js/`: http://codespeak.net/svn/pypy/branch/oo-jit/pypy/translator/js/

pypy/doc/_ref.txt

-.. _`demo/`: ../../demo
-.. _`demo/pickle_coroutine.py`: ../../demo/pickle_coroutine.py
-.. _`lib-python/`: ../../lib-python
-.. _`lib-python/2.5.2/dis.py`: ../../lib-python/2.5.2/dis.py
-.. _`annotation/`:
-.. _`pypy/annotation`: ../../pypy/annotation
-.. _`pypy/annotation/annrpython.py`: ../../pypy/annotation/annrpython.py
-.. _`annotation/binaryop.py`: ../../pypy/annotation/binaryop.py
-.. _`pypy/annotation/builtin.py`: ../../pypy/annotation/builtin.py
-.. _`pypy/annotation/model.py`: ../../pypy/annotation/model.py
-.. _`bin/`: ../../pypy/bin
-.. _`config/`: ../../pypy/config
-.. _`pypy/config/pypyoption.py`: ../../pypy/config/pypyoption.py
-.. _`doc/`: ../../pypy/doc
-.. _`doc/config/`: ../../pypy/doc/config
-.. _`doc/discussion/`: ../../pypy/doc/discussion
-.. _`interpreter/`:
-.. _`pypy/interpreter`: ../../pypy/interpreter
-.. _`pypy/interpreter/argument.py`: ../../pypy/interpreter/argument.py
-.. _`interpreter/astcompiler/`:
-.. _`pypy/interpreter/astcompiler`: ../../pypy/interpreter/astcompiler
-.. _`pypy/interpreter/executioncontext.py`: ../../pypy/interpreter/executioncontext.py
-.. _`pypy/interpreter/function.py`: ../../pypy/interpreter/function.py
-.. _`interpreter/gateway.py`:
-.. _`pypy/interpreter/gateway.py`: ../../pypy/interpreter/gateway.py
-.. _`pypy/interpreter/generator.py`: ../../pypy/interpreter/generator.py
-.. _`pypy/interpreter/mixedmodule.py`: ../../pypy/interpreter/mixedmodule.py
-.. _`pypy/interpreter/module.py`: ../../pypy/interpreter/module.py
-.. _`pypy/interpreter/nestedscope.py`: ../../pypy/interpreter/nestedscope.py
-.. _`pypy/interpreter/pyopcode.py`: ../../pypy/interpreter/pyopcode.py
-.. _`interpreter/pyparser/`:
-.. _`pypy/interpreter/pyparser`: ../../pypy/interpreter/pyparser
-.. _`pypy/interpreter/pyparser/pytokenizer.py`: ../../pypy/interpreter/pyparser/pytokenizer.py
-.. _`pypy/interpreter/pyparser/parser.py`: ../../pypy/interpreter/pyparser/parser.py
-.. _`pypy/interpreter/pyparser/pyparse.py`: ../../pypy/interpreter/pyparser/pyparse.py
-.. _`pypy/interpreter/pyparser/future.py`: ../../pypy/interpreter/pyparser/future.py
-.. _`pypy/interpreter/pyparser/metaparser.py`: ../../pypy/interpreter/pyparser/metaparser.py
-.. _`pypy/interpreter/astcompiler/astbuilder.py`: ../../pypy/interpreter/astcompiler/astbuilder.py
-.. _`pypy/interpreter/astcompiler/optimize.py`: ../../pypy/interpreter/astcompiler/optimize.py
-.. _`pypy/interpreter/astcompiler/codegen.py`: ../../pypy/interpreter/astcompiler/codegen.py
-.. _`pypy/interpreter/astcompiler/tools/asdl_py.py`: ../../pypy/interpreter/astcompiler/tools/asdl_py.py
-.. _`pypy/interpreter/astcompiler/tools/Python.asdl`: ../../pypy/interpreter/astcompiler/tools/Python.asdl
-.. _`pypy/interpreter/astcompiler/assemble.py`: ../../pypy/interpreter/astcompiler/assemble.py
-.. _`pypy/interpreter/astcompiler/symtable.py`: ../../pypy/interpreter/astcompiler/symtable.py
-.. _`pypy/interpreter/astcompiler/asthelpers.py`: ../../pypy/interpreter/astcompiler/asthelpers.py
-.. _`pypy/interpreter/astcompiler/ast.py`: ../../pypy/interpreter/astcompiler/ast.py
-.. _`pypy/interpreter/typedef.py`: ../../pypy/interpreter/typedef.py
-.. _`lib/`:
-.. _`lib_pypy/`: ../../lib_pypy
-.. _`lib/distributed/`: ../../lib_pypy/distributed
-.. _`lib_pypy/stackless.py`: ../../lib_pypy/stackless.py
-.. _`lib_pypy/pypy_test/`: ../../lib_pypy/pypy_test
-.. _`module/`:
-.. _`pypy/module`:
-.. _`pypy/module/`: ../../pypy/module
-.. _`pypy/module/__builtin__/__init__.py`: ../../pypy/module/__builtin__/__init__.py
-.. _`pypy/module/_stackless/test/test_clonable.py`: ../../pypy/module/_stackless/test/test_clonable.py
-.. _`pypy/module/_stackless/test/test_composable_coroutine.py`: ../../pypy/module/_stackless/test/test_composable_coroutine.py
-.. _`objspace/`:
-.. _`pypy/objspace`: ../../pypy/objspace
-.. _`objspace/dump.py`: ../../pypy/objspace/dump.py
-.. _`objspace/flow/`: ../../pypy/objspace/flow
-.. _`objspace/std/`:
-.. _`pypy/objspace/std`: ../../pypy/objspace/std
-.. _`objspace/taint.py`: ../../pypy/objspace/taint.py
-.. _`objspace/thunk.py`:
-.. _`pypy/objspace/thunk.py`: ../../pypy/objspace/thunk.py
-.. _`objspace/trace.py`:
-.. _`pypy/objspace/trace.py`: ../../pypy/objspace/trace.py
-.. _`pypy/rlib`:
-.. _`rlib/`: ../../pypy/rlib
-.. _`pypy/rlib/rarithmetic.py`: ../../pypy/rlib/rarithmetic.py
-.. _`pypy/rlib/test`: ../../pypy/rlib/test
-.. _`pypy/rpython`:
-.. _`pypy/rpython/`:
-.. _`rpython/`: ../../pypy/rpython
-.. _`rpython/lltypesystem/`: ../../pypy/rpython/lltypesystem
-.. _`pypy/rpython/lltypesystem/lltype.py`:
-.. _`rpython/lltypesystem/lltype.py`: ../../pypy/rpython/lltypesystem/lltype.py
-.. _`rpython/memory/`: ../../pypy/rpython/memory
-.. _`rpython/memory/gc/generation.py`: ../../pypy/rpython/memory/gc/generation.py
-.. _`rpython/memory/gc/hybrid.py`: ../../pypy/rpython/memory/gc/hybrid.py
-.. _`rpython/memory/gc/markcompact.py`: ../../pypy/rpython/memory/gc/markcompact.py
-.. _`rpython/memory/gc/marksweep.py`: ../../pypy/rpython/memory/gc/marksweep.py
-.. _`rpython/memory/gc/semispace.py`: ../../pypy/rpython/memory/gc/semispace.py
-.. _`rpython/ootypesystem/`: ../../pypy/rpython/ootypesystem
-.. _`rpython/ootypesystem/ootype.py`: ../../pypy/rpython/ootypesystem/ootype.py
-.. _`rpython/rint.py`: ../../pypy/rpython/rint.py
-.. _`rpython/rlist.py`: ../../pypy/rpython/rlist.py
-.. _`rpython/rmodel.py`: ../../pypy/rpython/rmodel.py
-.. _`pypy/rpython/rtyper.py`: ../../pypy/rpython/rtyper.py
-.. _`pypy/rpython/test/test_llinterp.py`: ../../pypy/rpython/test/test_llinterp.py
-.. _`pypy/test_all.py`: ../../pypy/test_all.py
-.. _`tool/`: ../../pypy/tool
-.. _`tool/algo/`: ../../pypy/tool/algo
-.. _`tool/pytest/`: ../../pypy/tool/pytest
-.. _`pypy/translator`:
-.. _`translator/`: ../../pypy/translator
-.. _`translator/backendopt/`: ../../pypy/translator/backendopt
-.. _`translator/c/`: ../../pypy/translator/c
-.. _`translator/cli/`: ../../pypy/translator/cli
-.. _`translator/goal/`: ../../pypy/translator/goal
-.. _`pypy/translator/goal/targetnopstandalone.py`: ../../pypy/translator/goal/targetnopstandalone.py
-.. _`translator/jvm/`: ../../pypy/translator/jvm
-.. _`translator/stackless/`: ../../pypy/translator/stackless
-.. _`translator/tool/`: ../../pypy/translator/tool
-.. _`translator/js/`: http://codespeak.net/svn/pypy/branch/oo-jit/pypy/translator/js/

pypy/doc/architecture.rst

+==================================================
+PyPy - Goals and Architecture Overview 
+==================================================
+
+.. contents::
+
+
+This document gives an overview of the goals and architecture of PyPy.
+See `getting started`_ for a practical introduction and starting points. 
+
+Mission statement 
+====================
+
+We aim to provide:
+
+ * a common translation and support framework for producing
+   implementations of dynamic languages, emphasizing a clean
+   separation between language specification and implementation
+   aspects.
+
+ * a compliant, flexible and fast implementation of the Python_ Language 
+   using the above framework to enable new advanced features without having
+   to encode low level details into it.
+
+By separating concerns in this way, we intend for our implementation
+of Python - and other dynamic languages - to become robust against almost 
+all implementation decisions, including target platform, memory and 
+threading models, optimizations applied, up to to the point of being able to
+automatically *generate* Just-in-Time compilers for dynamic languages.
+
+Conversely, our implementation techniques, including the JIT compiler 
+generator, should become robust against changes in the languages 
+implemented. 
+
+
+High Level Goals
+=============================
+
+PyPy - the Translation Framework 
+-----------------------------------------------
+
+Traditionally, language interpreters are written in a target platform language
+like C/Posix, Java or C#.  Each such implementation fundamentally provides 
+a mapping from application source code to the target environment.  One of 
+the goals of the "all-encompassing" environments, like the .NET framework
+and to some extent the Java virtual machine, is to provide standardized
+and higher level functionalities in order to support language implementers
+for writing language implementations. 
+
+PyPy is experimenting with a more ambitious approach.  We are using a
+subset of the high-level language Python, called RPython_, in which we
+write languages as simple interpreters with few references to and
+dependencies on lower level details.  Our translation framework then
+produces a concrete virtual machine for the platform of our choice by
+inserting appropriate lower level aspects.  The result can be customized
+by selecting other feature and platform configurations.
+
+Our goal is to provide a possible solution to the problem of language
+implementers: having to write ``l * o * p`` interpreters for ``l``
+dynamic languages and ``p`` platforms with ``o`` crucial design
+decisions.  PyPy aims at having any one of these parameters changeable
+independently from each other:
+
+* ``l``: the language that we analyze can be evolved or entirely replaced;
+
+* ``o``: we can tweak and optimize the translation process to produce 
+  platform specific code based on different models and trade-offs;
+
+* ``p``: we can write new translator back-ends to target different
+  physical and virtual platforms.
+
+By contrast, a standardized target environment - say .NET -
+enforces ``p=1`` as far as it's concerned.  This helps making ``o`` a
+bit smaller by providing a higher-level base to build upon.  Still,
+we believe that enforcing the use of one common environment 
+is not necessary.  PyPy's goal is to give weight to this claim - at least 
+as far as language implementation is concerned - showing an approach
+to the ``l * o * p`` problem that does not rely on standardization.
+
+The most ambitious part of this goal is to `generate Just-In-Time
+Compilers`_ in a language-independent way, instead of only translating
+the source interpreter into an interpreter for the target platform.
+This is an area of language implementation that is commonly considered
+very challenging because of the involved complexity.
+
+
+PyPy - the Python Interpreter 
+--------------------------------------------
+
+Our main motivation for developing the translation framework is to
+provide a full featured, customizable, fast_ and `very compliant`_ Python
+implementation, working on and interacting with a large variety of
+platforms and allowing the quick introduction of new advanced language
+features.
+
+This Python implementation is written in RPython as a relatively simple
+interpreter, in some respects easier to understand than CPython, the C
+reference implementation of Python.  We are using its high level and
+flexibility to quickly experiment with features or implementation
+techniques in ways that would, in a traditional approach, require
+pervasive changes to the source code.  For example, PyPy's Python
+interpreter can optionally provide lazily computed objects - a small
+extension that would require global changes in CPython.  Another example
+is the garbage collection technique: changing CPython to use a garbage
+collector not based on reference counting would be a major undertaking,
+whereas in PyPy it is an issue localized in the translation framework,
+and fully orthogonal to the interpreter source code.
+
+
+PyPy Architecture 
+===========================
+
+As you would expect from a project implemented using ideas from the world
+of `Extreme Programming`_, the architecture of PyPy has evolved over time
+and continues to evolve.  Nevertheless, the high level architecture is 
+stable. As described above, there are two rather independent basic
+subsystems: the `Python Interpreter`_ and the `Translation Framework`_.
+
+.. _`translation framework`:
+
+The Translation Framework
+-------------------------
+
+The job of the translation tool chain is to translate RPython_ programs
+into an efficient version of that program for one of various target
+platforms, generally one that is considerably lower-level than Python.
+
+The approach we have taken is to reduce the level of abstraction of the
+source RPython program in several steps, from the high level down to the
+level of the target platform, whatever that may be.  Currently we
+support two broad flavours of target platforms: the ones that assume a
+C-like memory model with structures and pointers, and the ones that
+assume an object-oriented model with classes, instances and methods (as,
+for example, the Java and .NET virtual machines do).
+
+The translation tool chain never sees the RPython source code or syntax
+trees, but rather starts with the *code objects* that define the
+behaviour of the function objects one gives it as input.  It can be
+considered as "freezing" a pre-imported RPython program into an
+executable form suitable for the target platform.
+
+The steps of the translation process can be summarized as follows:
+
+* The code object of each source functions is converted to a `control
+  flow graph` by the `Flow Object Space`_.
+
+* The control flow graphs are processed by the Annotator_, which
+  performs whole-program type inference to annotate each variable of
+  the control flow graph with the types it may take at run-time.
+
+* The information provided by the annotator is used by the RTyper_ to
+  convert the high level operations of the control flow graphs into
+  operations closer to the abstraction level of the target platform.
+
+* Optionally, `various transformations`_ can then be applied which, for
+  example, perform optimizations such as inlining, add capabilities
+  such as stackless_-style concurrency, or insert code for the
+  `garbage collector`_.
+
+* Then, the graphs are converted to source code for the target platform
+  and compiled into an executable.
+
+This process is described in much more detail in the `document about
+the translation process`_ and in the paper `Compiling dynamic language
+implementations`_.
+
+.. _`control flow graph`: translation.html#the-flow-model
+.. _`Flow Object Space`: objspace.html#the-flow-object-space
+.. _Annotator: translation.html#the-annotation-pass
+.. _RTyper: rtyper.html#overview
+.. _`various transformations`: translation.html#the-optional-transformations
+.. _`document about the translation process`: translation.html
+.. _`garbage collector`: garbage_collection.html
+
+
+.. _`standard interpreter`: 
+.. _`python interpreter`: 
+
+The Python Interpreter
+-------------------------------------
+
+PyPy's *Python Interpreter* is written in RPython and implements the
+full Python language.  This interpreter very closely emulates the
+behavior of CPython.  It contains the following key components:
+
+- a bytecode compiler responsible for producing Python code objects 
+  from the source code of a user application;
+
+- a `bytecode evaluator`_ responsible for interpreting 
+  Python code objects;
+
+- a `standard object space`_, responsible for creating and manipulating
+  the Python objects seen by the application.
+
+The *bytecode compiler* is the preprocessing phase that produces a
+compact bytecode format via a chain of flexible passes (tokenizer,
+lexer, parser, abstract syntax tree builder, bytecode generator).  The
+*bytecode evaluator* interprets this bytecode.  It does most of its work
+by delegating all actual manipulations of user objects to the *object
+space*.  The latter can be thought of as the library of built-in types.
+It defines the implementation of the user objects, like integers and
+lists, as well as the operations between them, like addition or
+truth-value-testing.
+
+This division between bytecode evaluator and object space is very
+important, as it gives a lot of flexibility.  One can plug in 
+different `object spaces`_ to get different or enriched behaviours 
+of the Python objects.  Additionally, a special more abstract object
+space, the `flow object space`_, allows us to reuse the bytecode
+evaluator for our translation framework.
+
+.. _`bytecode evaluator`: interpreter.html
+.. _`standard object space`: objspace.html#the-standard-object-space
+.. _`object spaces`: objspace.html
+.. _`flow object space`: objspace.html#the-flow-object-space
+
+.. _`the translation framework`:
+
+
+Further reading
+===============
+
+All of PyPy's documentation can be reached from the `documentation
+index`_.  Of particular interest after reading this document might be:
+
+ * `getting-started`_: a hands-on guide to getting involved with the
+   PyPy source code.
+
+ * `PyPy's approach to virtual machine construction`_: a paper
+   presented to the Dynamic Languages Symposium attached to OOPSLA
+   2006.
+
+ * `The translation document`_: a detailed description of our
+   translation process.
+
+ * All our `Technical reports`_, including `Compiling dynamic language
+   implementations`_.
+
+ * `JIT Generation in PyPy`_, describing how we produce a Just-in-time
+   Compiler from an interpreter.
+
+.. _`documentation index`: docindex.html
+.. _`getting-started`: getting-started.html
+.. _`PyPy's approach to virtual machine construction`: http://codespeak.net/svn/pypy/extradoc/talk/dls2006/pypy-vm-construction.pdf
+.. _`the translation document`: translation.html
+.. _`Compiling dynamic language implementations`: http://codespeak.net/svn/pypy/extradoc/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf
+.. _`Technical reports`: index-report.html
+
+.. _`getting started`: getting-started.html
+.. _`Extreme Programming`: http://www.extremeprogramming.org/
+
+.. _fast: faq.html#how-fast-is-pypy
+.. _`very compliant`: cpython_differences.html
+
+.. _`RPython`: coding-guide.html#rpython
+
+.. _Python: http://docs.python.org/ref
+.. _Psyco: http://psyco.sourceforge.net
+.. _stackless: stackless.html
+.. _`generate Just-In-Time Compilers`: jit/index.html
+.. _`JIT Generation in PyPy`: jit/index.html
+
+.. include:: _ref.txt
+

pypy/doc/architecture.txt

-==================================================
-PyPy - Goals and Architecture Overview 
-==================================================
-
-.. contents::
-
-
-This document gives an overview of the goals and architecture of PyPy.
-See `getting started`_ for a practical introduction and starting points. 
-
-Mission statement 
-====================
-
-We aim to provide:
-
- * a common translation and support framework for producing
-   implementations of dynamic languages, emphasizing a clean
-   separation between language specification and implementation
-   aspects.
-
- * a compliant, flexible and fast implementation of the Python_ Language 
-   using the above framework to enable new advanced features without having
-   to encode low level details into it.
-
-By separating concerns in this way, we intend for our implementation
-of Python - and other dynamic languages - to become robust against almost 
-all implementation decisions, including target platform, memory and 
-threading models, optimizations applied, up to to the point of being able to
-automatically *generate* Just-in-Time compilers for dynamic languages.
-
-Conversely, our implementation techniques, including the JIT compiler 
-generator, should become robust against changes in the languages 
-implemented. 
-
-
-High Level Goals
-=============================
-
-PyPy - the Translation Framework 
------------------------------------------------
-
-Traditionally, language interpreters are written in a target platform language
-like C/Posix, Java or C#.  Each such implementation fundamentally provides 
-a mapping from application source code to the target environment.  One of 
-the goals of the "all-encompassing" environments, like the .NET framework
-and to some extent the Java virtual machine, is to provide standardized
-and higher level functionalities in order to support language implementers
-for writing language implementations. 
-
-PyPy is experimenting with a more ambitious approach.  We are using a
-subset of the high-level language Python, called RPython_, in which we
-write languages as simple interpreters with few references to and
-dependencies on lower level details.  Our translation framework then
-produces a concrete virtual machine for the platform of our choice by
-inserting appropriate lower level aspects.  The result can be customized
-by selecting other feature and platform configurations.
-
-Our goal is to provide a possible solution to the problem of language
-implementers: having to write ``l * o * p`` interpreters for ``l``
-dynamic languages and ``p`` platforms with ``o`` crucial design
-decisions.  PyPy aims at having any one of these parameters changeable
-independently from each other:
-
-* ``l``: the language that we analyze can be evolved or entirely replaced;
-
-* ``o``: we can tweak and optimize the translation process to produce 
-  platform specific code based on different models and trade-offs;
-
-* ``p``: we can write new translator back-ends to target different
-  physical and virtual platforms.
-
-By contrast, a standardized target environment - say .NET -
-enforces ``p=1`` as far as it's concerned.  This helps making ``o`` a
-bit smaller by providing a higher-level base to build upon.  Still,
-we believe that enforcing the use of one common environment 
-is not necessary.  PyPy's goal is to give weight to this claim - at least 
-as far as language implementation is concerned - showing an approach
-to the ``l * o * p`` problem that does not rely on standardization.
-
-The most ambitious part of this goal is to `generate Just-In-Time
-Compilers`_ in a language-independent way, instead of only translating
-the source interpreter into an interpreter for the target platform.
-This is an area of language implementation that is commonly considered
-very challenging because of the involved complexity.
-
-
-PyPy - the Python Interpreter 
---------------------------------------------
-
-Our main motivation for developing the translation framework is to
-provide a full featured, customizable, fast_ and `very compliant`_ Python
-implementation, working on and interacting with a large variety of
-platforms and allowing the quick introduction of new advanced language
-features.
-
-This Python implementation is written in RPython as a relatively simple
-interpreter, in some respects easier to understand than CPython, the C
-reference implementation of Python.  We are using its high level and
-flexibility to quickly experiment with features or implementation
-techniques in ways that would, in a traditional approach, require
-pervasive changes to the source code.  For example, PyPy's Python
-interpreter can optionally provide lazily computed objects - a small
-extension that would require global changes in CPython.  Another example
-is the garbage collection technique: changing CPython to use a garbage
-collector not based on reference counting would be a major undertaking,
-whereas in PyPy it is an issue localized in the translation framework,
-and fully orthogonal to the interpreter source code.
-
-
-PyPy Architecture 
-===========================
-
-As you would expect from a project implemented using ideas from the world
-of `Extreme Programming`_, the architecture of PyPy has evolved over time
-and continues to evolve.  Nevertheless, the high level architecture is 
-stable. As described above, there are two rather independent basic
-subsystems: the `Python Interpreter`_ and the `Translation Framework`_.
-
-.. _`translation framework`:
-
-The Translation Framework
--------------------------
-
-The job of the translation tool chain is to translate RPython_ programs
-into an efficient version of that program for one of various target
-platforms, generally one that is considerably lower-level than Python.
-
-The approach we have taken is to reduce the level of abstraction of the
-source RPython program in several steps, from the high level down to the
-level of the target platform, whatever that may be.  Currently we
-support two broad flavours of target platforms: the ones that assume a
-C-like memory model with structures and pointers, and the ones that
-assume an object-oriented model with classes, instances and methods (as,
-for example, the Java and .NET virtual machines do).
-
-The translation tool chain never sees the RPython source code or syntax
-trees, but rather starts with the *code objects* that define the
-behaviour of the function objects one gives it as input.  It can be
-considered as "freezing" a pre-imported RPython program into an
-executable form suitable for the target platform.
-
-The steps of the translation process can be summarized as follows:
-
-* The code object of each source functions is converted to a `control
-  flow graph` by the `Flow Object Space`_.
-
-* The control flow graphs are processed by the Annotator_, which
-  performs whole-program type inference to annotate each variable of
-  the control flow graph with the types it may take at run-time.
-
-* The information provided by the annotator is used by the RTyper_ to
-  convert the high level operations of the control flow graphs into
-  operations closer to the abstraction level of the target platform.
-
-* Optionally, `various transformations`_ can then be applied which, for
-  example, perform optimizations such as inlining, add capabilities
-  such as stackless_-style concurrency, or insert code for the
-  `garbage collector`_.
-
-* Then, the graphs are converted to source code for the target platform
-  and compiled into an executable.
-
-This process is described in much more detail in the `document about
-the translation process`_ and in the paper `Compiling dynamic language
-implementations`_.
-
-.. _`control flow graph`: translation.html#the-flow-model
-.. _`Flow Object Space`: objspace.html#the-flow-object-space
-.. _Annotator: translation.html#the-annotation-pass
-.. _RTyper: rtyper.html#overview
-.. _`various transformations`: translation.html#the-optional-transformations
-.. _`document about the translation process`: translation.html
-.. _`garbage collector`: garbage_collection.html
-
-
-.. _`standard interpreter`: 
-.. _`python interpreter`: 
-
-The Python Interpreter
--------------------------------------
-
-PyPy's *Python Interpreter* is written in RPython and implements the
-full Python language.  This interpreter very closely emulates the
-behavior of CPython.  It contains the following key components:
-
-- a bytecode compiler responsible for producing Python code objects 
-  from the source code of a user application;
-
-- a `bytecode evaluator`_ responsible for interpreting 
-  Python code objects;
-
-- a `standard object space`_, responsible for creating and manipulating
-  the Python objects seen by the application.
-
-The *bytecode compiler* is the preprocessing phase that produces a
-compact bytecode format via a chain of flexible passes (tokenizer,
-lexer, parser, abstract syntax tree builder, bytecode generator).  The
-*bytecode evaluator* interprets this bytecode.  It does most of its work
-by delegating all actual manipulations of user objects to the *object
-space*.  The latter can be thought of as the library of built-in types.
-It defines the implementation of the user objects, like integers and
-lists, as well as the operations between them, like addition or
-truth-value-testing.
-
-This division between bytecode evaluator and object space is very
-important, as it gives a lot of flexibility.  One can plug in 
-different `object spaces`_ to get different or enriched behaviours 
-of the Python objects.  Additionally, a special more abstract object
-space, the `flow object space`_, allows us to reuse the bytecode
-evaluator for our translation framework.
-
-.. _`bytecode evaluator`: interpreter.html
-.. _`standard object space`: objspace.html#the-standard-object-space
-.. _`object spaces`: objspace.html
-.. _`flow object space`: objspace.html#the-flow-object-space
-
-.. _`the translation framework`:
-
-
-Further reading
-===============
-
-All of PyPy's documentation can be reached from the `documentation
-index`_.  Of particular interest after reading this document might be:
-
- * `getting-started`_: a hands-on guide to getting involved with the
-   PyPy source code.
-
- * `PyPy's approach to virtual machine construction`_: a paper
-   presented to the Dynamic Languages Symposium attached to OOPSLA
-   2006.
-
- * `The translation document`_: a detailed description of our
-   translation process.
-
- * All our `Technical reports`_, including `Compiling dynamic language
-   implementations`_.
-
- * `JIT Generation in PyPy`_, describing how we produce a Just-in-time
-   Compiler from an interpreter.
-
-.. _`documentation index`: docindex.html
-.. _`getting-started`: getting-started.html
-.. _`PyPy's approach to virtual machine construction`: http://codespeak.net/svn/pypy/extradoc/talk/dls2006/pypy-vm-construction.pdf
-.. _`the translation document`: translation.html
-.. _`Compiling dynamic language implementations`: http://codespeak.net/svn/pypy/extradoc/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf
-.. _`Technical reports`: index-report.html
-
-.. _`getting started`: getting-started.html
-.. _`Extreme Programming`: http://www.extremeprogramming.org/
-
-.. _fast: faq.html#how-fast-is-pypy
-.. _`very compliant`: cpython_differences.html
-
-.. _`RPython`: coding-guide.html#rpython
-
-.. _Python: http://docs.python.org/ref
-.. _Psyco: http://psyco.sourceforge.net
-.. _stackless: stackless.html
-.. _`generate Just-In-Time Compilers`: jit/index.html
-.. _`JIT Generation in PyPy`: jit/index.html
-
-.. include:: _ref.txt
-

pypy/doc/buildtool.rst

+============
+PyPyBuilder
+============
+
+What is this?
+=============
+
+PyPyBuilder is an application that allows people to build PyPy instances on
+demand. If you have a nice idle machine connected to the Internet, and don't
+mind us 'borrowing' it every once in a while, you can start up the client
+script (in bin/client) and have the server send compile jobs to your machine.
+If someone requests a build of PyPy that is not already available on the PyPy
+website, and your machine is capable of making such a build, the server may ask
+your machine to create it. If enough people participate, with diverse enough
+machines, a 'build farm' is created.
+
+Quick usage instructions
+========================
+
+For the impatient, that just want to get started, some quick instructions.
+
+First you'll need to have a checkout of the 'buildtool' package, that can
+be found here::
+
+  https://codespeak.net/svn/pypy/build/buildtool
+
+To start a compilation, run (from the buildtool root directory)::
+
+  $ ./bin/startcompile.py [options] <email address>
+
+where the options can be found by using --help, and the email address will be
+used to send mail to once the compilation is finished.
+
+To start a build server, to participate in the build farm, do::
+
+  $ ./bin/buildserver.py
+
+That's it for the compilation script and build server, if you have your own
+project and want to set up your own meta server, you'll have to be a bit more
+patient and read the details below...
+
+Components
+==========
+
+The application consists of 3 main components: a meta server component, a
+client component that handles compilations (let's call this a 'build server')
+and a small client component to start compile jobs (which we'll call
+'requesting clients' for now).
+
+The server waits for build server to register, and for compile job
+requests. When participating clients register, they pass the server information
+about what compilations the system can handle (system info), and a set of
+options to use for compilation (compile info).
+
+When now a requesting client requests a compilation job, the server checks
+whether a suitable binary is already available based on the system and compile
+info, and if so returns that. If there isn't one, the server walks through a
+list of connected participating clients to see if one of them can handle the
+job, and if so dispatches the compilation. If there's no participating client
+to handle the job, it gets queued until there is.
+
+If a client crashes during compilation, the build is restarted, or error
+information is sent to the logs and requesting client, depending on the type of
+error. As long as no compilation error occurs (read: on disconnects, system
+errors, etc.) compilation will be retried until a build is available.
+
+Once a build is available, the server will send an email to all clients waiting
+for the build (it could be that more than one person asked for some build at
+the same time!).
+
+Configuration
+=============
+
+There are several aspects to configuration on this system. Of course, for the
+meta server, build server and startcompile components there is configuration
+for the host and port to connect to, and there is some additional configuration
+for things like which mailhost to use (only applies to the server), but also
+there is configuration data passed around to determine what client is picked,
+and what the client needs to compile exactly.
+
+Config file
+-----------
+
+The host/port configuration etc. can be found in the file 'config.py' in the
+build tool dir. There are several things that can be configured here, mostly
+related to what application to build, and where to build it. Please read the
+file carefully when setting up a new build network, or when participating for
+compilation, because certain items (e.g. the svnpath_to_url function, or the
+client_checkers) can make the system a lot less secure when not configured
+properly.
+
+Note that all client-related configuration is done from command-line switches,
+so the configuration file is supposed to be changed on a per-project basis:
+unless you have specific needs, use a test version of the build tool, or are
+working on another project than PyPy, you will not want to modify the it.
+
+System configuration
+--------------------
+
+This information is used by the client and startcompile components. On the
+participating clients this information is retrieved by querying the system, on
+the requesting clients the system values are used by default, but may be
+overridden (so a requesting client running an x86 can still request PPC builds,
+for instance). The clients compare their own system config to that of a build
+request, and will (should) refuse a build if it can not be executed because
+of incompatibilities.
+
+Compilation configuration
+-------------------------
+
+The third form of configuration is that of the to-be-built application itself,
+its compilation arguments. This configuration is only provided by the
+requesting clients, build servers can examine the information and refuse a
+compilation based on this configuration (just like with the system config, see
+'client_checkers' in 'config.py'). Compilation configuration can be controlled
+using command-line arguments (use 'bin/startcompile.py --help' for an
+overview).
+
+Build tool options
+------------------
+
+Yet another part of the configuration are the options that are used by the
+startcompile.py script itself: the user can specify what SVN path (relative to
+a certain base path) and what Subversion revision is desired.  The revision can
+either be specified exactly, or as a range of versions.
+
+Installation
+============
+
+Build Server
+------------
+
+Installing the system should not be required: just run './bin/buildserver' to
+start. Note that it depends on the `py lib`_ (as does the rest of PyPy).
+
+When starting a build server with PyPy's default configuration, it will connect
+to a meta server we have running in codespeak.net.
+
+Meta Server
+-----------
+
+Also for the server there's no real setup required, and again there's a 
+dependency on the `py lib`_. Starting it is done by running
+'./bin/metaserver'.
+
+Running a compile job
+---------------------
+
+Again installation is not required, just run './bin/startcompile.py [options]
+<email>' (see --help for the options) to start. Again, you need to have the
+`py lib`_ installed.
+
+Normally the codespeak.net meta server will be used when this script is issued.
+
+.. _`py lib`: http://codespeak.net/py
+
+Using the build tool for other projects
+=======================================
+
+The code for the build tool is meant to be generic. Using it for other projects
+than PyPy (for which it was originally written) is relatively straight-forward:
+just change the configuration, and implement a build client script (probably
+highly resembling bin/buildserver.py).
+
+Note that there is a test project in 'tool/build/testproject' that can serve
+as an example.
+
+Prerequisites
+--------------
+
+Your project can use the build tool if:
+
+  * it can be built from Python
+
+    Of course this is a rather vague requirement: theoretically _anything_ can
+    be built from Python; it's just a matter of integrating it into the tool
+    properly... A project that can entirely be built from Python code (like
+    PyPy) is easier to integrate than something that is built from the command
+    line, though (although implementing that won't be very hard either, see
+    the test project for instance).
+
+  * it is located in Subversion
+
+    The build tool makes very little hard-coded assumptions, but having code
+    in Subversion is one of them. There are several locations in the code where
+    SVN is assumed: the command line options (see `build tool options`_),
+    the server (which checks SVN urls for validity, and converts HEAD revision
+    requests to actual revision ids) and and build client (which checks out the
+    data) all make this assumption, changing to a different revision control
+    system is currently not easy and unsupported (but who knows what the future
+    will bring).
+
+  * it uses PyPy's config mechanism
+
+    PyPy has a very nice, generic configuration mechanism (essentially wrapper
+    OptionParser stuff) that makes dealing with fragmented configuration
+    and command-line options a lot easier. This mechanism is used by the build
+    tool: it assumes configuration is provided in this format. If your project
+    uses this configuration mechanism already, you can provide the root Config
+    object from config.compile_config; if not it should be fairly straight-
+    forward to wrap your existing configuration with the PyPy stuff.
+
+Basically that's it: if your project is stored in SVN, and you don't mind using
+Python a bit, it shouldn't be too hard to get things going (note that more
+documentation about this subject will follow in the future).
+
+Web Front-End
+=============
+
+To examine the status of the meta server, connected build servers and build
+requests, there is a web server available. This can be started using
+'./bin/webserver' and uses port 8080 by default (override in
+config.py).
+
+The web server presents a number of different pages:
+
+  * / and /metaserverstatus - meta server status
+
+    this displays a small list of information about the meta server, such
+    as the amount of connected build servers, the amount of builds available,
+    the amount of waiting clients, etc.
+
+  * /buildservers - connected build servers
+
+    this page contains a list of all connected build servers, system
+    information and what build they're currently working on (if any)
+
+  * /builds - a list of builds
+
+    here you'll find a list of all builds, both done and in-progress and
+    queued ones, with links to the details pages, the date they were
+    requested and their status
+
+  * /build/<id> - build details
+
+    the 'build' (virtual) directory contains pages of information for each
+    build - each of those pages displays status information, time requested,
+    time started and finished (if appropriate), links to the zip and logs,
+    and system and compile information
+
+There's a build tool status web server for the meta server on codespeak.net
+available at http://codespeak.net/pypy/buildstatus/.
+
+More info
+=========
+
+For more information, bug reports, patches, etc., please send an email to 
+guido@merlinux.de.
+

pypy/doc/buildtool.txt

-============
-PyPyBuilder
-============
-
-What is this?
-=============
-
-PyPyBuilder is an application that allows people to build PyPy instances on
-demand. If you have a nice idle machine connected to the Internet, and don't
-mind us 'borrowing' it every once in a while, you can start up the client
-script (in bin/client) and have the server send compile jobs to your machine.
-If someone requests a build of PyPy that is not already available on the PyPy
-website, and your machine is capable of making such a build, the server may ask
-your machine to create it. If enough people participate, with diverse enough
-machines, a 'build farm' is created.
-
-Quick usage instructions
-========================
-
-For the impatient, that just want to get started, some quick instructions.
-
-First you'll need to have a checkout of the 'buildtool' package, that can
-be found here::
-
-  https://codespeak.net/svn/pypy/build/buildtool
-
-To start a compilation, run (from the buildtool root directory)::
-
-  $ ./bin/startcompile.py [options] <email address>
-
-where the options can be found by using --help, and the email address will be
-used to send mail to once the compilation is finished.
-
-To start a build server, to participate in the build farm, do::
-
-  $ ./bin/buildserver.py
-
-That's it for the compilation script and build server, if you have your own
-project and want to set up your own meta server, you'll have to be a bit more
-patient and read the details below...
-
-Components
-==========
-
-The application consists of 3 main components: a meta server component, a
-client component that handles compilations (let's call this a 'build server')
-and a small client component to start compile jobs (which we'll call
-'requesting clients' for now).
-
-The server waits for build server to register, and for compile job
-requests. When participating clients register, they pass the server information
-about what compilations the system can handle (system info), and a set of
-options to use for compilation (compile info).
-
-When now a requesting client requests a compilation job, the server checks
-whether a suitable binary is already available based on the system and compile
-info, and if so returns that. If there isn't one, the server walks through a
-list of connected participating clients to see if one of them can handle the
-job, and if so dispatches the compilation. If there's no participating client
-to handle the job, it gets queued until there is.
-
-If a client crashes during compilation, the build is restarted, or error
-information is sent to the logs and requesting client, depending on the type of
-error. As long as no compilation error occurs (read: on disconnects, system
-errors, etc.) compilation will be retried until a build is available.
-
-Once a build is available, the server will send an email to all clients waiting
-for the build (it could be that more than one person asked for some build at
-the same time!).
-
-Configuration
-=============
-
-There are several aspects to configuration on this system. Of course, for the
-meta server, build server and startcompile components there is configuration
-for the host and port to connect to, and there is some additional configuration
-for things like which mailhost to use (only applies to the server), but also
-there is configuration data passed around to determine what client is picked,
-and what the client needs to compile exactly.
-
-Config file
------------
-
-The host/port configuration etc. can be found in the file 'config.py' in the
-build tool dir. There are several things that can be configured here, mostly
-related to what application to build, and where to build it. Please read the
-file carefully when setting up a new build network, or when participating for
-compilation, because certain items (e.g. the svnpath_to_url function, or the
-client_checkers) can make the system a lot less secure when not configured
-properly.
-
-Note that all client-related configuration is done from command-line switches,
-so the configuration file is supposed to be changed on a per-project basis:
-unless you have specific needs, use a test version of the build tool, or are
-working on another project than PyPy, you will not want to modify the it.
-
-System configuration
---------------------
-
-This information is used by the client and startcompile components. On the
-participating clients this information is retrieved by querying the system, on
-the requesting clients the system values are used by default, but may be
-overridden (so a requesting client running an x86 can still request PPC builds,
-for instance). The clients compare their own system config to that of a build
-request, and will (should) refuse a build if it can not be executed because
-of incompatibilities.
-
-Compilation configuration
--------------------------
-
-The third form of configuration is that of the to-be-built application itself,
-its compilation arguments. This configuration is only provided by the
-requesting clients, build servers can examine the information and refuse a
-compilation based on this configuration (just like with the system config, see
-'client_checkers' in 'config.py'). Compilation configuration can be controlled
-using command-line arguments (use 'bin/startcompile.py --help' for an
-overview).
-
-Build tool options
-------------------
-
-Yet another part of the configuration are the options that are used by the
-startcompile.py script itself: the user can specify what SVN path (relative to
-a certain base path) and what Subversion revision is desired.  The revision can
-either be specified exactly, or as a range of versions.
-
-Installation
-============
-
-Build Server
-------------
-
-Installing the system should not be required: just run './bin/buildserver' to
-start. Note that it depends on the `py lib`_ (as does the rest of PyPy).
-
-When starting a build server with PyPy's default configuration, it will connect
-to a meta server we have running in codespeak.net.
-
-Meta Server
------------
-
-Also for the server there's no real setup required, and again there's a 
-dependency on the `py lib`_. Starting it is done by running
-'./bin/metaserver'.
-
-Running a compile job
----------------------
-
-Again installation is not required, just run './bin/startcompile.py [options]
-<email>' (see --help for the options) to start. Again, you need to have the
-`py lib`_ installed.
-
-Normally the codespeak.net meta server will be used when this script is issued.
-
-.. _`py lib`: http://codespeak.net/py
-
-Using the build tool for other projects
-=======================================
-
-The code for the build tool is meant to be generic. Using it for other projects
-than PyPy (for which it was originally written) is relatively straight-forward:
-just change the configuration, and implement a build client script (probably
-highly resembling bin/buildserver.py).
-
-Note that there is a test project in 'tool/build/testproject' that can serve
-as an example.
-
-Prerequisites
---------------
-
-Your project can use the build tool if:
-
-  * it can be built from Python
-
-    Of course this is a rather vague requirement: theoretically _anything_ can
-    be built from Python; it's just a matter of integrating it into the tool
-    properly... A project that can entirely be built from Python code (like
-    PyPy) is easier to integrate than something that is built from the command
-    line, though (although implementing that won't be very hard either, see
-    the test project for instance).
-
-  * it is located in Subversion
-
-    The build tool makes very little hard-coded assumptions, but having code
-    in Subversion is one of them. There are several locations in the code where
-    SVN is assumed: the command line options (see `build tool options`_),
-    the server (which checks SVN urls for validity, and converts HEAD revision
-    requests to actual revision ids) and and build client (which checks out the
-    data) all make this assumption, changing to a different revision control
-    system is currently not easy and unsupported (but who knows what the future
-    will bring).
-
-  * it uses PyPy's config mechanism
-
-    PyPy has a very nice, generic configuration mechanism (essentially wrapper
-    OptionParser stuff) that makes dealing with fragmented configuration
-    and command-line options a lot easier. This mechanism is used by the build
-    tool: it assumes configuration is provided in this format. If your project
-    uses this configuration mechanism already, you can provide the root Config
-    object from config.compile_config; if not it should be fairly straight-
-    forward to wrap your existing configuration with the PyPy stuff.
-
-Basically that's it: if your project is stored in SVN, and you don't mind using
-Python a bit, it shouldn't be too hard to get things going (note that more