Source

astoptimizer / README

Full commit
+++++++++++++
AST Optimizer
+++++++++++++

Experimental AST optimizer for Python.

Examples of optimized code:

 * 1 + 2 * 3 => 7
 * print(1.5) => print("1.5")
 * int("3") * float("2) => 6.0
 * (None,) * 4 => (None, None, None, None)
 * frozenset("ab") | frozenset("bc") => frozenset("abc")
 * "abcdef"[:3] => "abc"
 * len("abcdef") => 6
 * ord("A") => 65
 * 4 and 5 and x and 6 => x and 6
 * if debug: print("log") => pass (with debug declared as constant and equals
   to 0 or False)
 * while 0: code => pass
 * u"hého".encode("utf-8") => b'h\xc3\xa9ho'
 * (32).bit_length() => 6
 * float.fromhex("0x1.8000000000000p+0") => 1.5
 * math.log(32) / math.log(2) => 5.0

You can specified your own constants and your own pure functions for more
optimizations.


Limitations
===========

 * Builtin functions are only called at compile time if arguments are immutable
 * Non portable optimizations are disabled by default: len("\U0010ffff") is not
   optimized for example. Enable pythonenv and platform features to enable more
   (non portable) optimizations.
 * len(str) is not called at compile time if the compile time if the result is
   bigger than 2^31-1. Enable pythonenv to allow this optimization.


Features
========

By default, the optimizer only enables portable optimizations. If you control
your environment (platform, Python command line options, environment
variables), you can enable more optimizations using the Config() class.

Example enabling optimization for a specific platform and a specific Python
version::

    from astoptimizer import optimize_code
    config = Config('platform', 'pythonenv')
    exec(optimize_code(code, config=config))

All configuration features:

 - "builtins" (always enabled): enable optimization on builtin functions
 - "math" (always enabled): replace math constants by their value (ex: math.pi)
   and call math functions at compile time (ex: math.log)
 - "pythonenv": optimizations specific to the running Python executable.
   Don't enable these optimizations if you don't control the Python version,
   command line arguments and environment variables.
 - "platform": optimizations specific to a platform.


Example
=======

Example with the high-level function ``optimize_code``::

    from astoptimizer import optimize_code
    code = 'print(1+1)'
    code = optimize_code(code)
    exec(code)

Example the low-level functions ``optimize_ast``::

    from astoptimizer import Config, parse_ast, optimize_ast
    config = Config()
    code = 'print(1+1)'
    tree = parse_ast(code)
    tree = optimize_ast(tree, config)
    code = compile_ast(tree)
    exec(code)

See also demo.py script.


ChangeLog
=========

Version 0.2 (2012-09-02)
------------------------

Major changes:

 * Check input arguments before calling an operator or a function, instead of
   catching errors.
 * New helper functions optimize_code() and optimize_ast() should be used
   instead of using directly the Optimizer class.
 * Support tuple and frozenset types

Changes:

 * FIX: add Config.max_size to check len(obj) result
 * FIX: disable non portable optimizations on non-BMP strings
 * Support Python 2.5-3.3
 * Refactor Optimizer: Optimizer.visit() now always visit children before
   calling the optimizer for a node, except for assignments
 * Float and complex numbers are no more restricted by the integer range of the
   configuration
 * More builtin functions. Examples: divmod(int, int), float(str), min(tuple),
   sum(tuple).
 * More method of builtin types. Examples: str.startswith(), str.find(),
   tuple.count(), float.is_integer().
 * math module: add math.ceil(), math.floor() and math.trunc().
 * More module constants. Examples: os.O_RDONLY, errno.EINVAL,
   socket.SOCK_STREAM.
 * More operators: a not in b, a is b, a is not b, +a.
 * Conversion to string: str(), str % args and print(arg1, arg2, ...).
 * Support import aliases. Examples: "import math as M; print(M.floor(1.5))"
   and "from math import floor as F; print(F(1.5))".
 * Experimental support of variables (disabled by default).

Version 0.1 (2012-08-12)
------------------------

 * First public version (to reserve the name on PyPI!)


See also
========

http://pypi.python.org/pypi/asp

http://code.google.com/p/byteplay/

http://wiki.python.org/moin/PythonSpeed/PerformanceTips/

http://code.google.com/p/unladen-swallow/wiki/ProjectPlan

AST optimizer of PyPy:
https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/optimize.py

"A constant folding optimization pass for the AST"
http://bugs.python.org/issue1346238

"AST Optimization: inlining of function calls"
http://bugs.python.org/issue10399

"Build-out an AST optimizer, moving some functionality out of the peephole optimizer"
http://bugs.python.org/issue11549

http://geofft.mit.edu/blog/sipb/73


http://tomlee.co/2008/04/the-internals-of-python-generator-functions-in-the-ast/

http://pypi.python.org/pypi/BytecodeAssembler

http://wiki.python.org/moin/ByteplayDoc

http://www.slideshare.net/cjgiridhar/diving-into-byte-code-optimization-in-python

http://svn.python.org/view/python/branches/tlee-ast-optimize/Python/optimize.c?view=log
http://grokbase.com/p/python/python-dev/0853rf4s1a/ast-optimization-branch-elimination-in-generator-functions