1. Victor Stinner
  2. astoptimizer

Commits

Victor Stinner  committed dfb2c70

rename README to README.rst to render correctly README as reST

  • Participants
  • Parent commits 7dd03f2
  • Branches default

Comments (0)

Files changed (4)

File MANIFEST.in

View file
 include TODO
 include demo.py
 include run_tests.py
+include README.rst

File README

-+++++++++++++
-AST Optimizer
-+++++++++++++
-
-astoptimizer is an optimizer for Python code working on the Abstract Syntax
-Tree (AST, high-level representration). It does as much work as possible at
-compile time.
-
-The compiler is static, it is not a just-in-time (JIT) compiler, and so don't
-expect better performances than psyco or PyPy for example. Optimizations
-depending on the type of functions parameters cannot be done for examples.
-Only optimizations on immutable types (constants) are done.
-
-Website: http://pypi.python.org/pypi/astoptimizer
-
-Source code hosted at: https://bitbucket.org/haypo/astoptimizer
-
-
-Optimizations
-=============
-
- * Call builtin functions if arguments are constants (need "builtin_funcs"
-   feature). Examples:
-
-   - ``len("abc")`` => ``3``
-   - ``ord("A")`` => ``65``
-
- * Call methods of builtin types if the object and arguments are constants.
-   Examples:
-
-   - ``u"h\\xe9ho".encode("utf-8")`` => ``b"h\\xc3\\xa9ho"``
-   - ``"python2.7".startswith("python")`` => ``True``
-   - ``(32).bit_length()`` => ``6``
-   - ``float.fromhex("0x1.8p+0")`` => ``1.5``
-
- * Call functions of math and string modules for functions without
-   border effect. Examples:
-
-   - ``math.log(32) / math.log(2)`` => ``5.0``
-   - ``string.atoi("5")`` => ``5``
-
- * Format strings for str%args and print(arg1, arg2, ...) if arguments
-   are constants and the format string is valid.
-   Examples:
-
-   - ``"x=%s" % 5`` => ``"x=5"``
-   - ``print(1.5)`` => ``print("1.5")``
-
- * Simplify expressions. Examples:
-
-   - ``not(x in y)`` => ``x not in y``
-   - ``4 and 5 and x and 6`` => ``x and 6``
-   - ``if a: if b: print("true")`` => ``if a and b: print("true")``
-
- * Optimize loops (range => xrange needs "builtin_funcs" features). Examples:
-
-   - ``while True: pass`` => ``while 1: pass``
-   - ``for x in range(3): print(x)`` => ``x = 0; print(x); x = 1; print(x); x = 2; print(x)``
-   - ``for x in range(1000): print(x)`` => ``for x in xrange(1000): print(x)`` (Python 2)
-
- * Optimize iterators, list, set and dict comprehension, and generators (need
-   "builtin_funcs" feature). Examples:
-
-   - ``iter(set())`` => ``iter(())``
-   - ``frozenset("")`` => ``frozenset()``
-   - ``(x for x in "abc" if False)`` => ``(None for x in ())``
-   - ``[x*10 for x in range(1, 4)]`` => ``[10, 20, 30]``
-   - ``(x*2 for x in "abc" if True)`` => ``(x*2 for x in ("a", "b", "c"))``
-   - ``list(x for x in iterable)`` => ``list(iterable)``
-   - ``tuple(x for x in "abc")`` => ``("a", "b", "c")``
-   - ``list(x for x in range(3))`` => ``[0, 1, 2]``
-   - ``[x for x in ""]`` => ``[]``
-   - ``[x for x in iterable]`` => ``list(iterable)``
-   - ``set([x for x in "abc"])`` => ``{"a", "b", "c"}`` (Python 2.7+) or ``set(("a", "b", "c"))``
-
- * Replace list with tuple (need "builtin_funcs" feature). Examples:
-
-   - ``for x in [a, b, c]: print(x)`` => ``for x in (a, b, c): print(x)``
-   - ``x in [1, 2, 3]`` => ``x in (1, 2, 3)``
-   - ``list([x, y, z])`` => ``[x, y, z]``
-   - ``set([1, 2, 3])`` => ``{1, 2, 3}`` (Python 2.7+)
-
- * Evaluate unary and binary operators, subscript and comparaison if all
-   arguments are constants. Examples:
-
-   - ``1 + 2 * 3`` => ``7``
-   - ``not True`` => ``False``
-   - ``"abc" * 3`` => ``"abcabcabc"``
-   - ``"abcdef"[:3]`` => ``"abc"``
-   - ``(2, 7, 3)[1]`` => ``7``
-   - ``frozenset("ab") | frozenset("bc")`` => ``frozenset("abc")``
-   - ``None is None`` => ``True``
-   - ``"2" in "python2.7"`` => ``True``
-   - ``x in [1, 2, 3]`` => ``x in {1, 2, 3}`` (Python 3) or ``x in (1, 2, 3)`` (Python 2)
-   - ``def f(): return 2 if 4 < 5 else 3`` => ``def f(): return 2``
-
- * Remove dead code. Examples:
-
-   - ``def f(): return 1; return 2`` => ``def f(): return 1``
-   - ``def f(a, b): s = a+b; 3; return s`` => ``def f(a, b): s = a+b; return s``
-   - ``if DEBUG: print("debug")`` => ``pass`` with DEBUG declared as False
-   - ``while 0: print("never executed")`` => ``pass``
-
-
-Use astoptimizer in your project
-================================
-
-To enable astoptimizer globally on your project, add the following lines at the
-very begining of your application::
-
-    import astoptimizer
-    config = astoptimizer.Config('builtin_funcs', 'pythonbin')
-    # customize the config here
-    astoptimizer.patch_compile(config)
-
-On Python 3.3, imports will then use the patched compile() function and so
-all modules will be optimized. With older versions, the compileall module
-(ex: compileall.compile_dir()) can be used to compile an application
-with optimizations enabled.
-
-See also the issue `#17515: Add sys.setasthook() to allow to use
-a custom AST optimizer <http://bugs.python.org/issue17515>`_.
-
-
-Example
-=======
-
-Example with the high-level function ``optimize_code``::
-
-    from astoptimizer import optimize_code
-    code = "print(1+1)"
-    code = optimize_code(code)
-    exec(code)
-
-Example the low-level functions ``optimize_ast``::
-
-    from astoptimizer import Config, parse_ast, optimize_ast, compile_ast
-    config = Config('builtin_funcs', 'pythonbin')
-    code = "print(1+1)"
-    tree = parse_ast(code)
-    tree = optimize_ast(tree, config)
-    code = compile_ast(tree)
-    exec(code)
-
-See also ``demo.py`` script.
-
-
-Configuration
-=============
-
-Unsafe optimizations are disabled by default. Use the Config() class to enable
-more optimizations.
-
-Features enabled by default:
-
- * ``"builtin_types"``: methods of bytes, str, unicode, tuple, frozenset, int
-   and float types
- * ``"math"``, ``"string"``: constants and functions without border effects of
-   the math / string module
-
-Optional features:
-
- * ``"builtin_funcs"``: builtin functions like abs(), str(), len(), etc. Examples:
-
-   - ``len("abc")`` => ``3``
-   - ``ord("A")`` => ``65``
-   - ``str(123)`` => ``"123"``
-
- * ``"pythonbin"``: Enable this feature if the optimized code will be executed by
-   the same Python binary: so exactly the same Python version with the same
-   build options. Allow to optimize non-BMP unicode strings on Python < 3.3.
-   Enable the ``"platform"`` feature. Examples:
-
-   - ``u"\\U0010ffff"[0]`` => ``u"\\udbff"`` or ``u"\\U0010ffff"`` (depending on
-     build options, narrow or wide Unicode)
-   - ``sys.version_info.major`` => ``2``
-   - ``sys.maxunicode`` => ``0x10ffff``
-
- * ``"pythonenv"``: Enable this feature if you control the environment
-   variables (like ``PYTHONOPTIMIZE``) and Python command line options (like
-   ``-Qnew``).  On Python 2, allow to optimize int/int. Enable ``"platform"``
-   and ``"pythonbin"`` features. Examples:
-
-   - ``__debug__`` => ``True``
-   - ``sys.flags.optimize`` => ``0``
-
- * ``"platform"``: optimizations specific to a platform. Examples:
-
-   - ``sys.platform`` => ``"linux2"``
-   - ``sys.byteorder`` => ``"little"``
-   - ``sys.maxint`` => ``2147483647``
-   - ``os.linesep`` => ``"\\n"``
-
- * ``"struct"``: struct module, calcsize(), pack() and unpack() functions.
-
- * ``"cpython_tests"``: disable some optimizations to workaround issues with
-   the CPython test suite. Only use it for tests.
-
-Use ``Config("builtin_funcs", "pythonbin")`` to enable most optimizations.  You
-may also enable ``"pythonenv"`` to enable more optimizations, but then the
-optimized code will depends on environment variables and Python command line
-options.
-
-Use config.enable_all_optimizations() to enable all optimizations, which may
-generate invalid code.
-
-
-Advices
-=======
-
-Advices to help the AST optimizer:
-
- * Declare your constants using config.add_constant()
- * Declare your pure functions (functions with no border effect) using
-   config.add_func()
- * Don't use "from module import \*". If "import \*" is used, builtins
-   functions are not optimized anymore for example.
-
-
-Limitations
-===========
-
- * Operations on mutable values are not optimized, ex: len([1, 2, 3]).
- * Unsafe optimizations are disabled by default. For example, len("\\U0010ffff") is not
-   optimized because the result depends on the build options of Python. Enable
-   "builtin_funcs" and "pythonenv" features to enable more optimizations.
- * len() is not optimized if the result is bigger than 2^31-1.
-   Enable "pythonbin" configuration feature to optimize the call for bigger
-   objects.
- * On Python 2, operators taking a bytes string and a unicode string are not
-   optimized if the bytes string has to be decoded from the default encoding or
-   if the unicode string has to be encoded to the default encoding. Exception:
-   pure ASCII strings are optimized. For example, b"abc" + u"def" is replaced
-   with u"abcdef", whereas u"x=%s" % b"\\xe9" is not optimized.
- * On Python 3, comparaison between bytes and Unicode strings are not optimized
-   because the comparaison may emit a warning or raise a BytesWarning
-   exception. Bytes string are not converted to Unicode string. For example,
-   b"abc" < "abc" and str(b"abc") are not optimized. Converting a bytes string
-   to Unicode is never optimized.
-
-
-ChangeLog
-=========
-
-Version 0.6
------------
-
- * Remove empty loop. Example:
-   ``for i in (1, 2, 3): pass`` => ``i = 3``.
-
-Version 0.5 (2013-03-26)
-------------------------
-
- * Unroll loops (no support for break/continue yet) and list comprehension.
-   Example: ``[x*10 for x in range(1, 4)]`` => ``[10, 20, 30]``.
- * Add Config.enable_all_optimizations() method
- * Add a more aggressive option to remove dead code
-   (config.remove_almost_dead_code), disabled by default
- * Remove useless instructions. Example:
-   "x=1; 'abc'; print(x)" => "x=1; print(x)"
- * Remove empty try/except. Example:
-   "try: pass except: pass" => "pass"
-
-Version 0.4 (2012-12-10)
-------------------------
-
-Bugfixes:
-
- * Don't replace range() with xrange() if arguments cannot be converted to C
-   long
- * Disable float.fromhex() optimization by default: float may be shadowed.
-   Use "builtin_funcs" to enable this optimization.
-
-Changes:
-
- * Add the "struct" configuration feature: functions of the struct module
- * Optimize print() on Python 2 with "from __future__ import print_function"
- * Optimize iterators, list, set and dict comprehension, and generators
- * Replace list with tuple
- * Optimize ``if a: if b: print("true")``: ``if a and b: print("true")``
-
-Version 0.3.1 (2012-09-12)
---------------------------
-
-Bugfixes:
-
- * Disable optimizations on functions and constants if a variable with the same
-   name is set. Example: "len=ord; print(len('A'))",
-   "sys.version = 'abc'; print(sys.version)".
- * Don't optimize print() function, frozenset() nor range() functions if
-   "builtin_funcs" feature is disabled
- * Don't remove code if it contains global or nonlocal.
-   Example: "def f(): if 0: global x; x = 2".
-
-Version 0.3 (2012-09-11)
-------------------------
-
-Major changes:
-
- * Add astoptimizer.patch_compile(config=None) function to simply hook the
-   builtin compile() function.
- * Add "pythonbin" configuration feature.
- * Disable optimizations on builtin functions by default. Add "builtin_funcs"
-   feature to the configuration to optimize builtin functions.
- * Remove dead code (optionnal optimization)
- * It is now posible to define a callback for warnings of the optimizer
- * Drop support of Python 2.5, it is unable to compile an AST tree to bytecode.
-   AST objects of Python 2.5 don't accept arguments in constructors.
-
-Bugfixes:
-
- * Handle "from math import \*" correctly
- * Don't optimize operations if arguments are bytes and unicode strings.
-   Only optimize if string arguments have the same type.
- * Disable optimizations on non-BMP unicode strings by default. Optimizations
-   enabled with "pythonbin" feature.
-
-Other changes:
-
- * More functions, methods and constants:
-
-   - bytes, str, unicode: add more methods.
-   - math module: add most remaining functions
-   - string module: add some functions and all constants
-
- * not(a in b) => a not in b, not(a is b) => a is not b
- * a if bool else b
- * for x in range(n) => for x in xrange(n) (only on Python 2)
- * Enable more optimizations if a function is not a generator
- * Add sys.flags.<attr> and sys.version_info.<attr> constants
-
-Version 0.2 (2012-09-02)
-------------------------
-
-Major changes:
-
- * Check input arguments before calling an operator or a function, instead of
-   catching errors.
- * New helper functions optimize_code() and optimize_ast() should be used
-   instead of using directly the Optimizer class.
- * Support tuple and frozenset types
-
-Changes:
-
- * FIX: add Config.max_size to check len(obj) result
- * FIX: disable non portable optimizations on non-BMP strings
- * Support Python 2.5-3.3
- * Refactor Optimizer: Optimizer.visit() now always visit children before
-   calling the optimizer for a node, except for assignments
- * Float and complex numbers are no more restricted by the integer range of the
-   configuration
- * More builtin functions. Examples: divmod(int, int), float(str), min(tuple),
-   sum(tuple).
- * More method of builtin types. Examples: str.startswith(), str.find(),
-   tuple.count(), float.is_integer().
- * math module: add math.ceil(), math.floor() and math.trunc().
- * More module constants. Examples: os.O_RDONLY, errno.EINVAL,
-   socket.SOCK_STREAM.
- * More operators: a not in b, a is b, a is not b, +a.
- * Conversion to string: str(), str % args and print(arg1, arg2, ...).
- * Support import aliases. Examples: "import math as M; print(M.floor(1.5))"
-   and "from math import floor as F; print(F(1.5))".
- * Experimental support of variables (disabled by default).
-
-Version 0.1 (2012-08-12)
-------------------------
-
- * First public version (to reserve the name on PyPI!)
-
-
-See also
-========
-
-Faster Python implementations
------------------------------
-
- * `PyPy <http://pypy.org/>`_
-
-   - AST optimizer of PyPy:
-     `astcompiler/optimize.py <https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/optimize.py>`_
-
- * `Hotpy <http://code.google.com/p/hotpy/>`_
-   and `Hotpy 2 <https://bitbucket.org/markshannon/hotpy_2>`_,
-   based on `GVMT <http://code.google.com/p/gvmt/>`_ (Glasgow Virtual
-   Machine Toolkit)
- * `numba
-   <https://github.com/numba/numba>`_
- * `pymothoa <http://code.google.com/p/pymothoa/>`_ uses LLVM
-   ("don't support classes nor exceptions")
- * `WPython <http://code.google.com/p/wpython/>`_: 16-bit word-codes instead of byte-codes
- * `Cython <http://www.cython.org/>`_:
-
-   * `Compiler/Optimize.py
-     <https://github.com/cython/cython/blob/master/Cython/Compiler/Optimize.py>`_
-   * `Compiler/ParseTreeTransforms.py
-     <https://github.com/cython/cython/blob/master/Cython/Compiler/ParseTreeTransforms.py>`_
-   * `Compiler/Builtin.py
-     <https://github.com/cython/cython/blob/master/Cython/Compiler/Builtin.py>`_
-   * `Compiler/Pipeline.py
-     <https://github.com/cython/cython/blob/master/Cython/Compiler/Pipeline.py#L123>`_
-
-CPython issues
---------------
-
- * `Issue #1346238 <http://bugs.python.org/issue1346238>`_:
-   A constant folding optimization pass for the AST
- * `Issue #2181 <http://bugs.python.org/issue2181>`_:
-   optimize out local variables at end of function
- * `Issue #2499 <http://bugs.python.org/issue2499>`_:
-   Fold unary + and not on constants
- * `Issue #4264 <http://bugs.python.org/issue4264>`_:
-   Patch: optimize code to use LIST_APPEND instead of calling list.append
- * `Issue #7682 <http://bugs.python.org/issue7682>`_:
-   Optimisation of if with constant expression
- * `Issue #10399 <http://bugs.python.org/issue10399>`_:
-   AST Optimization: inlining of function calls
- * `Issue #11549 <http://bugs.python.org/issue11549>`_:
-   Build-out an AST optimizer, moving some functionality out of the peephole optimizer
- * `Issue #17068 <http://bugs.python.org/issue17068>`_:
-   peephole optimization for constant strings
- * `Issue #17430 <http://bugs.python.org/issue17430>`_:
-   missed peephole optimization
-
-Old projects
-------------
-
- * `Unladen Swallow <http://code.google.com/p/unladen-swallow/>`_, fork of
-   CPython 2.6, use LLVM. No more maintained
-
-   - `ProjectPlan
-     <http://code.google.com/p/unladen-swallow/wiki/ProjectPlan>`_
-   - `Unladen Swallow Retrospective
-     <http://qinsb.blogspot.com.au/2011/03/unladen-swallow-retrospective.html>`_
-   - `PEP 3146
-     <http://python.org/dev/peps/pep-3146/>`_
-
- * `psyco <http://psyco.sourceforge.net/>`_: JIT. The author of pysco, Armin
-   Rigo, co-created the PyPy project.
-
-AST
----
-
- * `instrumenting_the_ast.html <http://www.dalkescientific.com/writings/diary/archive/2010/02/22/instrumenting_the_ast.html>`_
- * `the-internals-of-python-generator-functions-in-the-ast
-   <http://tomlee.co/2008/04/the-internals-of-python-generator-functions-in-the-ast/>`_
- * `tlee-ast-optimize branch
-   <http://svn.python.org/view/python/branches/tlee-ast-optimize/Python/optimize.c?view=log>`_
- * `ast-optimization-branch-elimination-in-generator-functions
-   <http://grokbase.com/p/python/python-dev/0853rf4s1a/ast-optimization-branch-elimination-in-generator-functions>`_
-
-Bytecode
---------
-
- * `byteplay <http://code.google.com/p/byteplay/>`_
- * `diving-into-byte-code-optimization-in-python
-   <http://www.slideshare.net/cjgiridhar/diving-into-byte-code-optimization-in-python>`_
- * `BytecodeAssembler <http://pypi.python.org/pypi/BytecodeAssembler>`_
- * `ByteplayDoc <http://wiki.python.org/moin/ByteplayDoc>`_
- * `Hacking Python bytecode <http://geofft.mit.edu/blog/sipb/73>`_
-
-Other
------
-
- * `ASP <http://pypi.python.org/pypi/asp>`_:
-   ASP is a SEJITS (specialized embedded just-in-time compiler) toolkit for Python.
- * `PerformanceTips <http://wiki.python.org/moin/PythonSpeed/PerformanceTips/>`_
-

File README.rst

View file
++++++++++++++
+AST Optimizer
++++++++++++++
+
+astoptimizer is an optimizer for Python code working on the Abstract Syntax
+Tree (AST, high-level representration). It does as much work as possible at
+compile time.
+
+The compiler is static, it is not a just-in-time (JIT) compiler, and so don't
+expect better performances than psyco or PyPy for example. Optimizations
+depending on the type of functions parameters cannot be done for examples.
+Only optimizations on immutable types (constants) are done.
+
+Website: http://pypi.python.org/pypi/astoptimizer
+
+Source code hosted at: https://bitbucket.org/haypo/astoptimizer
+
+
+Optimizations
+=============
+
+ * Call builtin functions if arguments are constants (need "builtin_funcs"
+   feature). Examples:
+
+   - ``len("abc")`` => ``3``
+   - ``ord("A")`` => ``65``
+
+ * Call methods of builtin types if the object and arguments are constants.
+   Examples:
+
+   - ``u"h\\xe9ho".encode("utf-8")`` => ``b"h\\xc3\\xa9ho"``
+   - ``"python2.7".startswith("python")`` => ``True``
+   - ``(32).bit_length()`` => ``6``
+   - ``float.fromhex("0x1.8p+0")`` => ``1.5``
+
+ * Call functions of math and string modules for functions without
+   border effect. Examples:
+
+   - ``math.log(32) / math.log(2)`` => ``5.0``
+   - ``string.atoi("5")`` => ``5``
+
+ * Format strings for str%args and print(arg1, arg2, ...) if arguments
+   are constants and the format string is valid.
+   Examples:
+
+   - ``"x=%s" % 5`` => ``"x=5"``
+   - ``print(1.5)`` => ``print("1.5")``
+
+ * Simplify expressions. Examples:
+
+   - ``not(x in y)`` => ``x not in y``
+   - ``4 and 5 and x and 6`` => ``x and 6``
+   - ``if a: if b: print("true")`` => ``if a and b: print("true")``
+
+ * Optimize loops (range => xrange needs "builtin_funcs" features). Examples:
+
+   - ``while True: pass`` => ``while 1: pass``
+   - ``for x in range(3): print(x)`` => ``x = 0; print(x); x = 1; print(x); x = 2; print(x)``
+   - ``for x in range(1000): print(x)`` => ``for x in xrange(1000): print(x)`` (Python 2)
+
+ * Optimize iterators, list, set and dict comprehension, and generators (need
+   "builtin_funcs" feature). Examples:
+
+   - ``iter(set())`` => ``iter(())``
+   - ``frozenset("")`` => ``frozenset()``
+   - ``(x for x in "abc" if False)`` => ``(None for x in ())``
+   - ``[x*10 for x in range(1, 4)]`` => ``[10, 20, 30]``
+   - ``(x*2 for x in "abc" if True)`` => ``(x*2 for x in ("a", "b", "c"))``
+   - ``list(x for x in iterable)`` => ``list(iterable)``
+   - ``tuple(x for x in "abc")`` => ``("a", "b", "c")``
+   - ``list(x for x in range(3))`` => ``[0, 1, 2]``
+   - ``[x for x in ""]`` => ``[]``
+   - ``[x for x in iterable]`` => ``list(iterable)``
+   - ``set([x for x in "abc"])`` => ``{"a", "b", "c"}`` (Python 2.7+) or ``set(("a", "b", "c"))``
+
+ * Replace list with tuple (need "builtin_funcs" feature). Examples:
+
+   - ``for x in [a, b, c]: print(x)`` => ``for x in (a, b, c): print(x)``
+   - ``x in [1, 2, 3]`` => ``x in (1, 2, 3)``
+   - ``list([x, y, z])`` => ``[x, y, z]``
+   - ``set([1, 2, 3])`` => ``{1, 2, 3}`` (Python 2.7+)
+
+ * Evaluate unary and binary operators, subscript and comparaison if all
+   arguments are constants. Examples:
+
+   - ``1 + 2 * 3`` => ``7``
+   - ``not True`` => ``False``
+   - ``"abc" * 3`` => ``"abcabcabc"``
+   - ``"abcdef"[:3]`` => ``"abc"``
+   - ``(2, 7, 3)[1]`` => ``7``
+   - ``frozenset("ab") | frozenset("bc")`` => ``frozenset("abc")``
+   - ``None is None`` => ``True``
+   - ``"2" in "python2.7"`` => ``True``
+   - ``x in [1, 2, 3]`` => ``x in {1, 2, 3}`` (Python 3) or ``x in (1, 2, 3)`` (Python 2)
+   - ``def f(): return 2 if 4 < 5 else 3`` => ``def f(): return 2``
+
+ * Remove dead code. Examples:
+
+   - ``def f(): return 1; return 2`` => ``def f(): return 1``
+   - ``def f(a, b): s = a+b; 3; return s`` => ``def f(a, b): s = a+b; return s``
+   - ``if DEBUG: print("debug")`` => ``pass`` with DEBUG declared as False
+   - ``while 0: print("never executed")`` => ``pass``
+
+
+Use astoptimizer in your project
+================================
+
+To enable astoptimizer globally on your project, add the following lines at the
+very begining of your application::
+
+    import astoptimizer
+    config = astoptimizer.Config('builtin_funcs', 'pythonbin')
+    # customize the config here
+    astoptimizer.patch_compile(config)
+
+On Python 3.3, imports will then use the patched compile() function and so
+all modules will be optimized. With older versions, the compileall module
+(ex: compileall.compile_dir()) can be used to compile an application
+with optimizations enabled.
+
+See also the issue `#17515: Add sys.setasthook() to allow to use
+a custom AST optimizer <http://bugs.python.org/issue17515>`_.
+
+
+Example
+=======
+
+Example with the high-level function ``optimize_code``::
+
+    from astoptimizer import optimize_code
+    code = "print(1+1)"
+    code = optimize_code(code)
+    exec(code)
+
+Example the low-level functions ``optimize_ast``::
+
+    from astoptimizer import Config, parse_ast, optimize_ast, compile_ast
+    config = Config('builtin_funcs', 'pythonbin')
+    code = "print(1+1)"
+    tree = parse_ast(code)
+    tree = optimize_ast(tree, config)
+    code = compile_ast(tree)
+    exec(code)
+
+See also ``demo.py`` script.
+
+
+Configuration
+=============
+
+Unsafe optimizations are disabled by default. Use the Config() class to enable
+more optimizations.
+
+Features enabled by default:
+
+ * ``"builtin_types"``: methods of bytes, str, unicode, tuple, frozenset, int
+   and float types
+ * ``"math"``, ``"string"``: constants and functions without border effects of
+   the math / string module
+
+Optional features:
+
+ * ``"builtin_funcs"``: builtin functions like abs(), str(), len(), etc. Examples:
+
+   - ``len("abc")`` => ``3``
+   - ``ord("A")`` => ``65``
+   - ``str(123)`` => ``"123"``
+
+ * ``"pythonbin"``: Enable this feature if the optimized code will be executed by
+   the same Python binary: so exactly the same Python version with the same
+   build options. Allow to optimize non-BMP unicode strings on Python < 3.3.
+   Enable the ``"platform"`` feature. Examples:
+
+   - ``u"\\U0010ffff"[0]`` => ``u"\\udbff"`` or ``u"\\U0010ffff"`` (depending on
+     build options, narrow or wide Unicode)
+   - ``sys.version_info.major`` => ``2``
+   - ``sys.maxunicode`` => ``0x10ffff``
+
+ * ``"pythonenv"``: Enable this feature if you control the environment
+   variables (like ``PYTHONOPTIMIZE``) and Python command line options (like
+   ``-Qnew``).  On Python 2, allow to optimize int/int. Enable ``"platform"``
+   and ``"pythonbin"`` features. Examples:
+
+   - ``__debug__`` => ``True``
+   - ``sys.flags.optimize`` => ``0``
+
+ * ``"platform"``: optimizations specific to a platform. Examples:
+
+   - ``sys.platform`` => ``"linux2"``
+   - ``sys.byteorder`` => ``"little"``
+   - ``sys.maxint`` => ``2147483647``
+   - ``os.linesep`` => ``"\\n"``
+
+ * ``"struct"``: struct module, calcsize(), pack() and unpack() functions.
+
+ * ``"cpython_tests"``: disable some optimizations to workaround issues with
+   the CPython test suite. Only use it for tests.
+
+Use ``Config("builtin_funcs", "pythonbin")`` to enable most optimizations.  You
+may also enable ``"pythonenv"`` to enable more optimizations, but then the
+optimized code will depends on environment variables and Python command line
+options.
+
+Use config.enable_all_optimizations() to enable all optimizations, which may
+generate invalid code.
+
+
+Advices
+=======
+
+Advices to help the AST optimizer:
+
+ * Declare your constants using config.add_constant()
+ * Declare your pure functions (functions with no border effect) using
+   config.add_func()
+ * Don't use "from module import \*". If "import \*" is used, builtins
+   functions are not optimized anymore for example.
+
+
+Limitations
+===========
+
+ * Operations on mutable values are not optimized, ex: len([1, 2, 3]).
+ * Unsafe optimizations are disabled by default. For example, len("\\U0010ffff") is not
+   optimized because the result depends on the build options of Python. Enable
+   "builtin_funcs" and "pythonenv" features to enable more optimizations.
+ * len() is not optimized if the result is bigger than 2^31-1.
+   Enable "pythonbin" configuration feature to optimize the call for bigger
+   objects.
+ * On Python 2, operators taking a bytes string and a unicode string are not
+   optimized if the bytes string has to be decoded from the default encoding or
+   if the unicode string has to be encoded to the default encoding. Exception:
+   pure ASCII strings are optimized. For example, b"abc" + u"def" is replaced
+   with u"abcdef", whereas u"x=%s" % b"\\xe9" is not optimized.
+ * On Python 3, comparaison between bytes and Unicode strings are not optimized
+   because the comparaison may emit a warning or raise a BytesWarning
+   exception. Bytes string are not converted to Unicode string. For example,
+   b"abc" < "abc" and str(b"abc") are not optimized. Converting a bytes string
+   to Unicode is never optimized.
+
+
+ChangeLog
+=========
+
+Version 0.6
+-----------
+
+ * Remove empty loop. Example:
+   ``for i in (1, 2, 3): pass`` => ``i = 3``.
+
+Version 0.5 (2013-03-26)
+------------------------
+
+ * Unroll loops (no support for break/continue yet) and list comprehension.
+   Example: ``[x*10 for x in range(1, 4)]`` => ``[10, 20, 30]``.
+ * Add Config.enable_all_optimizations() method
+ * Add a more aggressive option to remove dead code
+   (config.remove_almost_dead_code), disabled by default
+ * Remove useless instructions. Example:
+   "x=1; 'abc'; print(x)" => "x=1; print(x)"
+ * Remove empty try/except. Example:
+   "try: pass except: pass" => "pass"
+
+Version 0.4 (2012-12-10)
+------------------------
+
+Bugfixes:
+
+ * Don't replace range() with xrange() if arguments cannot be converted to C
+   long
+ * Disable float.fromhex() optimization by default: float may be shadowed.
+   Use "builtin_funcs" to enable this optimization.
+
+Changes:
+
+ * Add the "struct" configuration feature: functions of the struct module
+ * Optimize print() on Python 2 with "from __future__ import print_function"
+ * Optimize iterators, list, set and dict comprehension, and generators
+ * Replace list with tuple
+ * Optimize ``if a: if b: print("true")``: ``if a and b: print("true")``
+
+Version 0.3.1 (2012-09-12)
+--------------------------
+
+Bugfixes:
+
+ * Disable optimizations on functions and constants if a variable with the same
+   name is set. Example: "len=ord; print(len('A'))",
+   "sys.version = 'abc'; print(sys.version)".
+ * Don't optimize print() function, frozenset() nor range() functions if
+   "builtin_funcs" feature is disabled
+ * Don't remove code if it contains global or nonlocal.
+   Example: "def f(): if 0: global x; x = 2".
+
+Version 0.3 (2012-09-11)
+------------------------
+
+Major changes:
+
+ * Add astoptimizer.patch_compile(config=None) function to simply hook the
+   builtin compile() function.
+ * Add "pythonbin" configuration feature.
+ * Disable optimizations on builtin functions by default. Add "builtin_funcs"
+   feature to the configuration to optimize builtin functions.
+ * Remove dead code (optionnal optimization)
+ * It is now posible to define a callback for warnings of the optimizer
+ * Drop support of Python 2.5, it is unable to compile an AST tree to bytecode.
+   AST objects of Python 2.5 don't accept arguments in constructors.
+
+Bugfixes:
+
+ * Handle "from math import \*" correctly
+ * Don't optimize operations if arguments are bytes and unicode strings.
+   Only optimize if string arguments have the same type.
+ * Disable optimizations on non-BMP unicode strings by default. Optimizations
+   enabled with "pythonbin" feature.
+
+Other changes:
+
+ * More functions, methods and constants:
+
+   - bytes, str, unicode: add more methods.
+   - math module: add most remaining functions
+   - string module: add some functions and all constants
+
+ * not(a in b) => a not in b, not(a is b) => a is not b
+ * a if bool else b
+ * for x in range(n) => for x in xrange(n) (only on Python 2)
+ * Enable more optimizations if a function is not a generator
+ * Add sys.flags.<attr> and sys.version_info.<attr> constants
+
+Version 0.2 (2012-09-02)
+------------------------
+
+Major changes:
+
+ * Check input arguments before calling an operator or a function, instead of
+   catching errors.
+ * New helper functions optimize_code() and optimize_ast() should be used
+   instead of using directly the Optimizer class.
+ * Support tuple and frozenset types
+
+Changes:
+
+ * FIX: add Config.max_size to check len(obj) result
+ * FIX: disable non portable optimizations on non-BMP strings
+ * Support Python 2.5-3.3
+ * Refactor Optimizer: Optimizer.visit() now always visit children before
+   calling the optimizer for a node, except for assignments
+ * Float and complex numbers are no more restricted by the integer range of the
+   configuration
+ * More builtin functions. Examples: divmod(int, int), float(str), min(tuple),
+   sum(tuple).
+ * More method of builtin types. Examples: str.startswith(), str.find(),
+   tuple.count(), float.is_integer().
+ * math module: add math.ceil(), math.floor() and math.trunc().
+ * More module constants. Examples: os.O_RDONLY, errno.EINVAL,
+   socket.SOCK_STREAM.
+ * More operators: a not in b, a is b, a is not b, +a.
+ * Conversion to string: str(), str % args and print(arg1, arg2, ...).
+ * Support import aliases. Examples: "import math as M; print(M.floor(1.5))"
+   and "from math import floor as F; print(F(1.5))".
+ * Experimental support of variables (disabled by default).
+
+Version 0.1 (2012-08-12)
+------------------------
+
+ * First public version (to reserve the name on PyPI!)
+
+
+See also
+========
+
+Faster Python implementations
+-----------------------------
+
+ * `PyPy <http://pypy.org/>`_
+
+   - AST optimizer of PyPy:
+     `astcompiler/optimize.py <https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/optimize.py>`_
+
+ * `Hotpy <http://code.google.com/p/hotpy/>`_
+   and `Hotpy 2 <https://bitbucket.org/markshannon/hotpy_2>`_,
+   based on `GVMT <http://code.google.com/p/gvmt/>`_ (Glasgow Virtual
+   Machine Toolkit)
+ * `numba
+   <https://github.com/numba/numba>`_
+ * `pymothoa <http://code.google.com/p/pymothoa/>`_ uses LLVM
+   ("don't support classes nor exceptions")
+ * `WPython <http://code.google.com/p/wpython/>`_: 16-bit word-codes instead of byte-codes
+ * `Cython <http://www.cython.org/>`_:
+
+   * `Compiler/Optimize.py
+     <https://github.com/cython/cython/blob/master/Cython/Compiler/Optimize.py>`_
+   * `Compiler/ParseTreeTransforms.py
+     <https://github.com/cython/cython/blob/master/Cython/Compiler/ParseTreeTransforms.py>`_
+   * `Compiler/Builtin.py
+     <https://github.com/cython/cython/blob/master/Cython/Compiler/Builtin.py>`_
+   * `Compiler/Pipeline.py
+     <https://github.com/cython/cython/blob/master/Cython/Compiler/Pipeline.py#L123>`_
+
+CPython issues
+--------------
+
+ * `Issue #1346238 <http://bugs.python.org/issue1346238>`_:
+   A constant folding optimization pass for the AST
+ * `Issue #2181 <http://bugs.python.org/issue2181>`_:
+   optimize out local variables at end of function
+ * `Issue #2499 <http://bugs.python.org/issue2499>`_:
+   Fold unary + and not on constants
+ * `Issue #4264 <http://bugs.python.org/issue4264>`_:
+   Patch: optimize code to use LIST_APPEND instead of calling list.append
+ * `Issue #7682 <http://bugs.python.org/issue7682>`_:
+   Optimisation of if with constant expression
+ * `Issue #10399 <http://bugs.python.org/issue10399>`_:
+   AST Optimization: inlining of function calls
+ * `Issue #11549 <http://bugs.python.org/issue11549>`_:
+   Build-out an AST optimizer, moving some functionality out of the peephole optimizer
+ * `Issue #17068 <http://bugs.python.org/issue17068>`_:
+   peephole optimization for constant strings
+ * `Issue #17430 <http://bugs.python.org/issue17430>`_:
+   missed peephole optimization
+
+Old projects
+------------
+
+ * `Unladen Swallow <http://code.google.com/p/unladen-swallow/>`_, fork of
+   CPython 2.6, use LLVM. No more maintained
+
+   - `ProjectPlan
+     <http://code.google.com/p/unladen-swallow/wiki/ProjectPlan>`_
+   - `Unladen Swallow Retrospective
+     <http://qinsb.blogspot.com.au/2011/03/unladen-swallow-retrospective.html>`_
+   - `PEP 3146
+     <http://python.org/dev/peps/pep-3146/>`_
+
+ * `psyco <http://psyco.sourceforge.net/>`_: JIT. The author of pysco, Armin
+   Rigo, co-created the PyPy project.
+
+AST
+---
+
+ * `instrumenting_the_ast.html <http://www.dalkescientific.com/writings/diary/archive/2010/02/22/instrumenting_the_ast.html>`_
+ * `the-internals-of-python-generator-functions-in-the-ast
+   <http://tomlee.co/2008/04/the-internals-of-python-generator-functions-in-the-ast/>`_
+ * `tlee-ast-optimize branch
+   <http://svn.python.org/view/python/branches/tlee-ast-optimize/Python/optimize.c?view=log>`_
+ * `ast-optimization-branch-elimination-in-generator-functions
+   <http://grokbase.com/p/python/python-dev/0853rf4s1a/ast-optimization-branch-elimination-in-generator-functions>`_
+
+Bytecode
+--------
+
+ * `byteplay <http://code.google.com/p/byteplay/>`_
+ * `diving-into-byte-code-optimization-in-python
+   <http://www.slideshare.net/cjgiridhar/diving-into-byte-code-optimization-in-python>`_
+ * `BytecodeAssembler <http://pypi.python.org/pypi/BytecodeAssembler>`_
+ * `ByteplayDoc <http://wiki.python.org/moin/ByteplayDoc>`_
+ * `Hacking Python bytecode <http://geofft.mit.edu/blog/sipb/73>`_
+
+Other
+-----
+
+ * `ASP <http://pypi.python.org/pypi/asp>`_:
+   ASP is a SEJITS (specialized embedded just-in-time compiler) toolkit for Python.
+ * `PerformanceTips <http://wiki.python.org/moin/PythonSpeed/PerformanceTips/>`_
+

File setup.py

View file
 # Todo list to prepare a release:
 #  - run: ./run_tests.py with Python 2.6, 2.7, 3.1, 3.2 and 3.3
 #  - edit astoptimizer/version.py: check/set version
-#  - edit README: set release date
+#  - edit README.rst: set release date
 #  - hg ci
 #  - hg tag astoptimizer-x.y
 #  - hg push
 #
 # After the release:
 #  - edit astoptimizer/version.py: set version to n+1
-#  - edit README: add a new empty section for version n+1
+#  - edit README.rst: add a new empty section for version n+1
 #  - hg ci
 #  - hg push
 
 
     metadata = load_source("version", path.join("astoptimizer", "version.py"))
 
-    with open('README') as fp:
+    with open('README.rst') as fp:
         long_description = fp.read()
 
     install_options = {