Commits

gstarnberger  committed 67e4cc5

rebranding 'unpyc' -> 'uncompyle'

  • Participants
  • Parent commits ac79174

Comments (0)

Files changed (73)

 
 Announcing:
 
-                 unpyc -- A Python byte-code decompiler
+                 uncompyle -- A Python byte-code decompiler
 
                       version 1.0
 
 
-'unpyc' converts Python byte-code back into equivalent Python
+'uncompyle' converts Python byte-code back into equivalent Python
 source. It accepts byte-code from any Python version starting with 1.5
 up to 2.7.
 
 The generated source is very readable: docstrings, lists, tuples and
 hashes get pretty-printed.
 
-'unpyc' may also verify the equivalence of the generated source by
+'uncompyle' may also verify the equivalence of the generated source by
 by compiling it and comparing both byte-codes.
 
 
   * Now decompiles byte-code from Python versions 1.5 up to 2.3
 
   * Use the marshal and dis code from each python version to make
-    decompilation independent of the interpreter unpyc runs with.
+    decompilation independent of the interpreter uncompyle runs with.
     (One can for example decompile python2.3 bytecode while running 
      under python 2.2)
 
 
   * pretty-prints docstrings, hashes, lists and tuples
 
-  * unpyc is now a script and a package
+  * uncompyle is now a script and a package
 
   * Now adds an emacs mode-hint and tab-width to the output files
 
 Simple Help Needed!
 -------------------
 
-Please help testing 'unpyc'! 
+Please help testing 'uncompyle'! 
 
 The EXTENDED_ARG token is untested (this is a new token for Python 2.0
 which is used only if many items exist within a code object). If you
 Requirements
 ------------
 
-'unpyc' requires Python 2.2 or later.
+'uncompyle' requires Python 2.2 or later.
 
 
 Known Bugs/Restrictions
 include MANIFEST.in 
 include README
 include ANNOUNCE CHANGES TODO
-include unpyc
+include uncompyle
 include test_pythonlib
 include test_one
 include compile_tests
 Metadata-Version: 1.1
-Name: unpyc
+Name: uncompyle
 Version: 1.1
 Summary: Python byte-code to source-code converter
-Home-page: http://code.google.com/p/unpyc/
+Home-page: http://github.com/sysfrog/uncompyle
 Author: Hartmut Goebel
 Author-email: hartmut@oberon.noris.de
 License: GPLv3
 
-                 unpyc -- A Python byte-code decompiler
+                 uncompyle -- A Python byte-code decompiler
                           1.1
                           2010-07-23
 
 Introduction
 ------------
 
-'unpyc' converts Python byte-code back into equivalent Python
+'uncompyle' converts Python byte-code back into equivalent Python
 source. It accepts byte-code from any Python version starting with 1.5
 up to 2.7.
 
 The generated source is very readable: docstrings, lists, tuples and
 hashes get pretty-printed.
 
-'unpyc' may also verify the equivalence of the generated source by
+'uncompyle' may also verify the equivalence of the generated source by
 by compiling it and comparing both byte-codes.
 
-'unpyc' is based on John Aycock's generic small languages compiler
+'uncompyle' is based on John Aycock's generic small languages compiler
 'spark' (http://www.csr.uvic.ca/~aycock/python/) and his prior work on
-'unpyc'.
+'uncompyle'.
 
 Additional note (3 July 2004, Ben Burton):
 
 Usage
 -----
 
-unpyc -h		prints short usage
-unpyc --help	prints long usage
+uncompyle -h		prints short usage
+uncompyle --help	prints long usage
 
 
 Known Bugs/Restrictions
 
 * Python 2.2 generated different byte-code than prior version for the
   same source. This is due the intruduction of iterators. Currently
-  'unpyc' fails verifying this source if the byte-code was
+  'uncompyle' fails verifying this source if the byte-code was
   generated by an older version of Python.

File scripts/uncompyle

+#!/usr/bin/python
+# Mode: -*- python -*-
+#
+# Copyright (c) 2000-2002 by hartmut Goebel <hartmut@goebel.noris.de>
+#
+"""
+Usage: uncompyle [OPTIONS]... [ FILE | DIR]...
+
+Examples:
+  uncompyle      foo.pyc bar.pyc       # uncompyle foo.pyc, bar.pyc to stdout
+  uncompyle -o . foo.pyc bar.pyc       # uncompyle to ./foo.dis and ./bar.dis
+  uncompyle -o /tmp /usr/lib/python1.5 # uncompyle whole library
+
+Options:
+  -o <path>     output decompiled files to this path:
+                if multiple input files are decompiled, the common prefix
+                is stripped from these names and the remainder appended to
+                <path>
+                  uncompyle -o /tmp bla/fasel.pyc bla/foo.pyc
+                    -> /tmp/fasel.dis, /tmp/foo.dis
+                  uncompyle -o /tmp bla/fasel.pyc bar/foo.pyc
+                    -> /tmp/bla/fasel.dis, /tmp/bar/foo.dis
+                  uncompyle -o /tmp /usr/lib/python1.5
+                    -> /tmp/smtplib.dis ... /tmp/lib-tk/FixTk.dis
+  -c <code>     attempts a disassembly after compiling <code
+  -d            do not print timestamps
+  --verify      compare generated source with input byte-code
+                (requires -o)
+  --help        show this message
+
+Debugging Options:
+  --showasm   -a  include byte-code                  (disables --verify)
+  --showast   -t  include AST (abstract syntax tree) (disables --verify)
+
+Extensions of generated files:
+  '.dis'             successfully decompiled (and verified if --verify)
+  '.dis_unverified'  successfully decompile but --verify failed
+  '.nodis'           uncompyle failed (contact author for enhancement)
+"""
+
+Usage_short = \
+"decomyple [--help] [--verify] [--showasm] [--showast] [-o <path>] FILE|DIR..."
+
+import sys, os, getopt
+import os.path
+from uncompyle import main, verify
+import time
+
+showasm = showast = do_verify = 0
+outfile = '-'
+out_base = None
+codes = []
+timestamp = True
+timestampfmt = "# %Y.%m.%d %H:%M:%S %Z"
+
+try:
+    opts, files = getopt.getopt(sys.argv[1:], 'hatdo:c:',
+                           ['help', 'verify', 'showast', 'showasm'])
+except getopt.GetoptError, e:
+    print >>sys.stderr, '%s: %s' % (os.path.basename(sys.argv[0]), e)
+    sys.exit(-1)    
+
+for opt, val in opts:
+    if opt in ('-h', '--help'):
+        print __doc__
+        sys.exit(0)
+    elif opt == '--verify':
+        do_verify = 1
+    elif opt in ('--showasm', '-a'):
+        showasm = 1
+        do_verify = 0
+    elif opt in ('--showast', '-t'):
+        showast = 1
+        do_verify = 0
+    elif opt == '-o':
+        outfile = val
+    elif opt == '-d':
+        timestamp = False
+    elif opt == '-c':
+        codes.append(val)
+    else:
+        print Usage_short
+        sys.exit(1)
+
+# argl, commonprefix works on strings, not on path parts,
+# thus we must handle the case with files in 'some/classes'
+# and 'some/cmds'
+src_base = os.path.commonprefix(files)
+if src_base[-1:] != os.sep:
+    src_base = os.path.dirname(src_base)
+if src_base:
+    sb_len = len( os.path.join(src_base, '') )
+    files = map(lambda f: f[sb_len:], files)
+    del sb_len
+    
+if outfile == '-':
+    outfile = None # use stdout
+elif outfile and os.path.isdir(outfile):
+    out_base = outfile; outfile = None
+elif outfile and len(files) > 1:
+    out_base = outfile; outfile = None
+
+if timestamp:
+    print time.strftime(timestampfmt)
+
+try:
+    main(src_base, out_base, files, codes, outfile, showasm, showast, do_verify)
+except KeyboardInterrupt, OSError:
+    pass
+except verify.VerifyCmpError:
+    raise
+
+if timestamp:
+    print time.strftime(timestampfmt)

File scripts/unpyc

-#!/usr/bin/python
-# Mode: -*- python -*-
-#
-# Copyright (c) 2000-2002 by hartmut Goebel <hartmut@goebel.noris.de>
-#
-"""
-Usage: unpyc [OPTIONS]... [ FILE | DIR]...
-
-Examples:
-  unpyc      foo.pyc bar.pyc       # unpyc foo.pyc, bar.pyc to stdout
-  unpyc -o . foo.pyc bar.pyc       # unpyc to ./foo.dis and ./bar.dis
-  unpyc -o /tmp /usr/lib/python1.5 # unpyc whole library
-
-Options:
-  -o <path>     output decompiled files to this path:
-                if multiple input files are decompiled, the common prefix
-                is stripped from these names and the remainder appended to
-                <path>
-                  unpyc -o /tmp bla/fasel.pyc bla/foo.pyc
-                    -> /tmp/fasel.dis, /tmp/foo.dis
-                  unpyc -o /tmp bla/fasel.pyc bar/foo.pyc
-                    -> /tmp/bla/fasel.dis, /tmp/bar/foo.dis
-                  unpyc -o /tmp /usr/lib/python1.5
-                    -> /tmp/smtplib.dis ... /tmp/lib-tk/FixTk.dis
-  -c <code>     attempts a disassembly after compiling <code
-  -d            do not print timestamps
-  --verify      compare generated source with input byte-code
-                (requires -o)
-  --help        show this message
-
-Debugging Options:
-  --showasm   -a  include byte-code                  (disables --verify)
-  --showast   -t  include AST (abstract syntax tree) (disables --verify)
-
-Extensions of generated files:
-  '.dis'             successfully decompiled (and verified if --verify)
-  '.dis_unverified'  successfully decompile but --verify failed
-  '.nodis'           unpyc failed (contact author for enhancement)
-"""
-
-Usage_short = \
-"decomyple [--help] [--verify] [--showasm] [--showast] [-o <path>] FILE|DIR..."
-
-import sys, os, getopt
-import os.path
-from unpyc import main, verify
-import time
-
-showasm = showast = do_verify = 0
-outfile = '-'
-out_base = None
-codes = []
-timestamp = True
-timestampfmt = "# %Y.%m.%d %H:%M:%S %Z"
-
-try:
-    opts, files = getopt.getopt(sys.argv[1:], 'hatdo:c:',
-                           ['help', 'verify', 'showast', 'showasm'])
-except getopt.GetoptError, e:
-    print >>sys.stderr, '%s: %s' % (os.path.basename(sys.argv[0]), e)
-    sys.exit(-1)    
-
-for opt, val in opts:
-    if opt in ('-h', '--help'):
-        print __doc__
-        sys.exit(0)
-    elif opt == '--verify':
-        do_verify = 1
-    elif opt in ('--showasm', '-a'):
-        showasm = 1
-        do_verify = 0
-    elif opt in ('--showast', '-t'):
-        showast = 1
-        do_verify = 0
-    elif opt == '-o':
-        outfile = val
-    elif opt == '-d':
-        timestamp = False
-    elif opt == '-c':
-        codes.append(val)
-    else:
-        print Usage_short
-        sys.exit(1)
-
-# argl, commonprefix works on strings, not on path parts,
-# thus we must handle the case with files in 'some/classes'
-# and 'some/cmds'
-src_base = os.path.commonprefix(files)
-if src_base[-1:] != os.sep:
-    src_base = os.path.dirname(src_base)
-if src_base:
-    sb_len = len( os.path.join(src_base, '') )
-    files = map(lambda f: f[sb_len:], files)
-    del sb_len
-    
-if outfile == '-':
-    outfile = None # use stdout
-elif outfile and os.path.isdir(outfile):
-    out_base = outfile; outfile = None
-elif outfile and len(files) > 1:
-    out_base = outfile; outfile = None
-
-if timestamp:
-    print time.strftime(timestampfmt)
-
-try:
-    main(src_base, out_base, files, codes, outfile, showasm, showast, do_verify)
-except KeyboardInterrupt, OSError:
-    pass
-except verify.VerifyCmpError:
-    raise
-
-if timestamp:
-    print time.strftime(timestampfmt)
 #!/usr/bin/env python
 
-"""Setup script for the 'unpyc' distribution."""
+"""Setup script for the 'uncompyle' distribution."""
 
 from distutils.core import setup, Extension
 
-setup (name = "unpyc",
+setup (name = "uncompyle",
        version = "1.1",
        description = "Python byte-code to source-code converter",
        author = "Hartmut Goebel",
        author_email = "hartmut@oberon.noris.de",
-       url = "http://code.google.com/p/unpyc/",
-       packages=['unpyc'],
-       scripts=['scripts/unpyc'],
-       ext_modules = [Extension('unpyc/marshal_20',
-                                ['unpyc/marshal_20.c'],
+       url = "http://github.com/sysfrog/uncompyle",
+       packages=['uncompyle'],
+       scripts=['scripts/uncompyle'],
+       ext_modules = [Extension('uncompyle/marshal_20',
+                                ['uncompyle/marshal_20.c'],
                                 define_macros=[]),
-                      Extension('unpyc/marshal_21',
-                                ['unpyc/marshal_21.c'],
+                      Extension('uncompyle/marshal_21',
+                                ['uncompyle/marshal_21.c'],
                                 define_macros=[]),
-                      Extension('unpyc/marshal_22',
-                                ['unpyc/marshal_22.c'],
+                      Extension('uncompyle/marshal_22',
+                                ['uncompyle/marshal_22.c'],
                                 define_macros=[]),
-                      Extension('unpyc/marshal_23',
-                                ['unpyc/marshal_23.c'],
+                      Extension('uncompyle/marshal_23',
+                                ['uncompyle/marshal_23.c'],
                                 define_macros=[]),
-                      Extension('unpyc/marshal_24',
-                                ['unpyc/marshal_24.c'],
+                      Extension('uncompyle/marshal_24',
+                                ['uncompyle/marshal_24.c'],
                                 define_macros=[]),
-                      Extension('unpyc/marshal_25',
-                                ['unpyc/marshal_25.c'],
+                      Extension('uncompyle/marshal_25',
+                                ['uncompyle/marshal_25.c'],
                                 define_macros=[]),
-                      Extension('unpyc/marshal_26',
-                                ['unpyc/marshal_26.c'],
+                      Extension('uncompyle/marshal_26',
+                                ['uncompyle/marshal_26.c'],
                                 define_macros=[]),
-                      Extension('unpyc/marshal_27',
-                                ['unpyc/marshal_27.c'],
+                      Extension('uncompyle/marshal_27',
+                                ['uncompyle/marshal_27.c'],
                                 define_macros=[])
                       ]
       )
     file=$BASEDIR/test_$file.pyc
 fi
 
-python -u ./scripts/unpyc $options $file 2>&1 |less
+python -u ./scripts/uncompyle $options $file 2>&1 |less

File test_pythonlib

 #!/usr/bin/env python
 # emacs-mode: -*-python-*-
 """
-test_pythonlib -- unpyc and verify Python libraries
+test_pythonlib -- uncompyle and verify Python libraries
 
 Usage-Examples:
 
 	  test_pythonlib --mylib --verify # decompile verify 'mylib'
 """
 
-from unpyc import main, verify
+from uncompyle import main, verify
 import os, time, shutil
 from fnmatch import fnmatch
 

File uncompyle/Parser.py

+#  Copyright (c) 1999 John Aycock
+#  Copyright (c) 2000-2002 by hartmut Goebel <hartmut@goebel.noris.de>
+#  Copyright (c) 2005 by Dan Pascu <dan@windowmaker.org>
+#
+#  See main module for license.
+#
+
+__all__ = ['parse', 'AST', 'ParserError', 'Parser']
+
+from spark import GenericASTBuilder
+import string, exceptions, sys
+from UserList import UserList
+
+from Scanner import Token
+
+class AST(UserList):
+    def __init__(self, type, kids=[]):
+        self.type = intern(type)
+        UserList.__init__(self, kids)
+
+    def __getslice__(self, low, high):    return self.data[low:high]
+    def __eq__(self, o):
+        if isinstance(o, AST):
+            return self.type == o.type \
+                   and UserList.__eq__(self, o)
+        else:
+            return self.type == o
+
+    def __hash__(self):            return hash(self.type)
+
+    def __repr__(self, indent=''):
+        rv = str(self.type)
+        for k in self:
+            rv = rv + '\n' + string.replace(str(k), '\n', '\n   ')
+        return rv
+
+
+class ParserError(Exception):
+    def __init__(self, token, offset):
+        self.token = token
+        self.offset = offset
+
+    def __str__(self):
+        return "Syntax error at or near `%r' token at offset %s" % \
+               (self.token, self.offset)
+    
+
+class Parser(GenericASTBuilder):
+    def __init__(self):
+        GenericASTBuilder.__init__(self, AST, 'stmts')
+        self.customized = {}
+
+    def cleanup(self):
+        """
+        Remove recursive references to allow garbage
+        collector to collect this object.
+        """
+        for dict in (self.rule2func, self.rules, self.rule2name, self.first):
+            for i in dict.keys():
+                dict[i] = None
+        for i in dir(self):
+            setattr(self, i, None)
+
+    def error(self, token):
+            raise ParserError(token, token.offset)
+
+    def typestring(self, token):
+        return token.type
+    
+    def p_funcdef(self, args):
+        '''
+        stmt ::= funcdef
+        funcdef ::= mkfunc designator
+        load_closure ::= load_closure LOAD_CLOSURE
+        load_closure ::= LOAD_CLOSURE
+        '''
+
+    def p_list_comprehension(self, args):
+        '''
+        expr ::= list_compr
+        list_compr ::= BUILD_LIST_0 DUP_TOP LOAD_ATTR
+                designator list_iter del_stmt
+
+        list_iter ::= list_for
+        list_iter ::= list_if
+        list_iter ::= lc_body
+
+        list_for ::= expr _for designator list_iter
+                JUMP_ABSOLUTE COME_FROM
+        list_if ::= expr condjmp list_iter
+                _jump COME_FROM POP_TOP
+                COME_FROM
+
+        lc_body ::= LOAD_NAME expr CALL_FUNCTION_1 POP_TOP
+        lc_body ::= LOAD_FAST expr CALL_FUNCTION_1 POP_TOP
+        '''
+        
+    def p_augmented_assign(self, args):
+        '''
+        stmt ::= augassign1
+        stmt ::= augassign2
+        augassign1 ::= expr expr inplace_op designator
+        augassign1 ::= expr expr inplace_op ROT_THREE STORE_SUBSCR
+        augassign1 ::= expr expr inplace_op ROT_TWO   STORE_SLICE+0
+        augassign1 ::= expr expr inplace_op ROT_THREE STORE_SLICE+1
+        augassign1 ::= expr expr inplace_op ROT_THREE STORE_SLICE+2
+        augassign1 ::= expr expr inplace_op ROT_FOUR  STORE_SLICE+3
+        augassign2 ::= expr DUP_TOP LOAD_ATTR expr
+                inplace_op ROT_TWO   STORE_ATTR
+
+        inplace_op ::= INPLACE_ADD
+        inplace_op ::= INPLACE_SUBTRACT
+        inplace_op ::= INPLACE_MULTIPLY
+        inplace_op ::= INPLACE_DIVIDE
+        inplace_op ::= INPLACE_TRUE_DIVIDE
+        inplace_op ::= INPLACE_FLOOR_DIVIDE
+        inplace_op ::= INPLACE_MODULO
+        inplace_op ::= INPLACE_POWER
+        inplace_op ::= INPLACE_LSHIFT
+        inplace_op ::= INPLACE_RSHIFT
+        inplace_op ::= INPLACE_AND
+        inplace_op ::= INPLACE_XOR
+        inplace_op ::= INPLACE_OR 
+        '''
+
+    def p_assign(self, args):
+        '''
+        stmt ::= assign
+        assign ::= expr DUP_TOP designList
+        assign ::= expr designator
+        '''
+
+    def p_print(self, args):
+        '''
+        stmt ::= print_stmt
+        stmt ::= print_stmt_nl
+        stmt ::= print_nl_stmt
+        print_stmt ::= expr PRINT_ITEM
+        print_nl_stmt ::= PRINT_NEWLINE
+        print_stmt_nl ::= print_stmt print_nl_stmt
+        '''
+
+    def p_print_to(self, args):
+        '''
+        stmt ::= print_to
+        stmt ::= print_to_nl
+        stmt ::= print_nl_to
+        print_to ::= expr print_to_items POP_TOP
+        print_to_nl ::= expr print_to_items PRINT_NEWLINE_TO
+        print_nl_to ::= expr PRINT_NEWLINE_TO
+        print_to_items ::= print_to_items print_to_item
+        print_to_items ::= print_to_item
+        print_to_item ::= DUP_TOP expr ROT_TWO PRINT_ITEM_TO
+        '''
+        # expr   print_to*   POP_TOP
+        # expr { print_to* } PRINT_NEWLINE_TO
+
+    def p_import15(self, args):
+        '''
+        stmt ::= importstmt
+        stmt ::= importfrom
+
+        importstmt ::= IMPORT_NAME STORE_FAST
+        importstmt ::= IMPORT_NAME STORE_NAME
+
+        importfrom ::= IMPORT_NAME importlist POP_TOP
+        importlist ::= importlist IMPORT_FROM
+        importlist ::= IMPORT_FROM
+        '''
+
+    def p_import20(self, args):
+        '''
+        stmt ::= importstmt2
+        stmt ::= importfrom2
+        stmt ::= importstar2
+        
+        stmt ::= _25_importstmt
+        stmt ::= _25_importfrom
+        stmt ::= _25_importstar
+
+        importstmt2 ::= LOAD_CONST import_as
+        importstar2 ::= LOAD_CONST IMPORT_NAME IMPORT_STAR
+
+        importfrom2 ::= LOAD_CONST IMPORT_NAME importlist2 POP_TOP
+        importlist2 ::= importlist2 import_as
+        importlist2 ::= import_as
+        import_as ::= IMPORT_NAME designator
+        import_as ::= IMPORT_NAME LOAD_ATTR designator
+        import_as ::= IMPORT_FROM designator
+
+        _25_importstmt ::= LOAD_CONST LOAD_CONST import_as 
+        _25_importstar ::= LOAD_CONST LOAD_CONST IMPORT_NAME IMPORT_STAR 
+        _25_importfrom ::= LOAD_CONST LOAD_CONST IMPORT_NAME importlist2 POP_TOP
+        '''
+
+    def p_grammar(self, args):
+        '''
+        stmts ::= stmts stmt
+        stmts ::= stmt
+
+        stmts_opt ::= stmts
+        stmts_opt ::= passstmt
+        passstmt ::= 
+
+        designList ::= designator designator
+        designList ::= designator DUP_TOP designList
+
+        designator ::= STORE_FAST
+        designator ::= STORE_NAME
+        designator ::= STORE_GLOBAL
+        designator ::= STORE_DEREF
+        designator ::= expr STORE_ATTR
+        designator ::= expr STORE_SLICE+0
+        designator ::= expr expr STORE_SLICE+1
+        designator ::= expr expr STORE_SLICE+2
+        designator ::= expr expr expr STORE_SLICE+3
+        designator ::= store_subscr
+        store_subscr ::= expr expr STORE_SUBSCR
+        designator ::= unpack
+        designator ::= unpack_list
+        
+        stmt ::= classdef
+        stmt ::= call_stmt
+        call_stmt ::= expr POP_TOP
+
+        stmt ::= return_stmt
+        return_stmt ::= expr RETURN_VALUE
+
+        stmt ::= yield_stmt
+        yield_stmt ::= expr YIELD_STMT
+        yield_stmt ::= expr YIELD_VALUE
+
+        stmt ::= break_stmt
+        break_stmt ::= BREAK_LOOP
+        
+        stmt ::= continue_stmt
+        continue_stmt ::= JUMP_ABSOLUTE
+        continue_stmt ::= CONTINUE_LOOP
+        
+        stmt ::= raise_stmt
+        raise_stmt ::= exprlist RAISE_VARARGS
+        raise_stmt ::= nullexprlist RAISE_VARARGS
+        
+        stmt ::= exec_stmt
+        exec_stmt ::= expr exprlist DUP_TOP EXEC_STMT
+        exec_stmt ::= expr exprlist EXEC_STMT
+        
+        stmt ::= assert
+        stmt ::= assert2
+        stmt ::= assert3
+        stmt ::= assert4
+        stmt ::= ifstmt
+        stmt ::= ifelsestmt
+        
+        stmt ::= _25_ifstmt
+        stmt ::= _25_ifelsestmt
+        
+        stmt ::= whilestmt
+        stmt ::= while1stmt
+        stmt ::= whileelsestmt
+        stmt ::= while1elsestmt
+        stmt ::= forstmt
+        stmt ::= forelsestmt
+        stmt ::= trystmt
+        stmt ::= tryfinallystmt
+        
+        stmt ::= del_stmt
+        del_stmt ::= DELETE_FAST
+        del_stmt ::= DELETE_NAME
+        del_stmt ::= DELETE_GLOBAL
+        del_stmt ::= expr DELETE_SLICE+0
+        del_stmt ::= expr expr DELETE_SLICE+1
+        del_stmt ::= expr expr DELETE_SLICE+2
+        del_stmt ::= expr expr expr DELETE_SLICE+3
+        del_stmt ::= delete_subscr
+        delete_subscr ::= expr expr DELETE_SUBSCR
+        del_stmt ::= expr DELETE_ATTR
+        
+        kwarg   ::= LOAD_CONST expr
+        
+        classdef ::= LOAD_CONST expr mkfunc
+                    CALL_FUNCTION_0 BUILD_CLASS designator
+
+        condjmp    ::= JUMP_IF_FALSE POP_TOP
+        condjmp    ::= JUMP_IF_TRUE  POP_TOP
+
+        assert ::= expr JUMP_IF_FALSE POP_TOP
+                expr JUMP_IF_TRUE POP_TOP
+                LOAD_GLOBAL RAISE_VARARGS
+                COME_FROM COME_FROM POP_TOP
+        assert2 ::= expr JUMP_IF_FALSE POP_TOP
+                expr JUMP_IF_TRUE POP_TOP
+                LOAD_GLOBAL expr RAISE_VARARGS
+                COME_FROM COME_FROM POP_TOP
+        assert3 ::= expr JUMP_IF_TRUE POP_TOP
+                LOAD_GLOBAL RAISE_VARARGS
+                COME_FROM POP_TOP
+        assert4 ::= expr JUMP_IF_TRUE POP_TOP
+                LOAD_GLOBAL expr RAISE_VARARGS
+                COME_FROM POP_TOP
+
+        stmt ::= ifforstmt
+        ifforstmt ::= expr condjmp SETUP_LOOP expr _for designator
+        stmts_opt JUMP_ABSOLUTE
+        COME_FROM POP_BLOCK _jump COME_FROM
+
+        _jump ::= JUMP_ABSOLUTE
+        _jump ::= JUMP_FORWARD
+
+        ifstmt ::= expr condjmp stmts_opt
+                _jump COME_FROM POP_TOP COME_FROM
+
+        ifelsestmt ::= expr condjmp stmts_opt
+                _jump COME_FROM
+                POP_TOP stmts COME_FROM
+                
+        _25_ifstmt ::= _25_bexp _25_jumptarget_e _jump _25_jumptarget_e COME_FROM
+        _25_ifelsestmt ::= _25_bexp _25_jumptarget_e _jump _25_jumptarget_s COME_FROM
+        
+        _25_jumptarget_e ::= _25_come_froms stmts_opt
+        _25_jumptarget_s ::= _25_come_froms stmts
+        
+        _25_come_froms ::= COME_FROM _25_come_froms
+        _25_come_froms ::= POP_TOP
+
+        trystmt ::= SETUP_EXCEPT stmts_opt
+                POP_BLOCK _jump
+                COME_FROM except_stmt
+
+        try_end  ::= END_FINALLY COME_FROM
+        try_end  ::= except_else
+        except_else ::= END_FINALLY COME_FROM stmts
+
+        except_stmt ::= except_cond except_stmt COME_FROM
+        except_stmt ::= except_conds try_end COME_FROM
+        except_stmt ::= except try_end COME_FROM
+        except_stmt ::= try_end
+
+        except_conds ::= except_cond except_conds COME_FROM
+        except_conds ::= 
+
+        except_cond ::= except_cond1
+        except_cond ::= except_cond2
+        except_cond1 ::= DUP_TOP expr COMPARE_OP
+                JUMP_IF_FALSE
+                POP_TOP POP_TOP POP_TOP POP_TOP
+                stmts_opt _jump COME_FROM
+                POP_TOP
+        except_cond2 ::= DUP_TOP expr COMPARE_OP
+                JUMP_IF_FALSE
+                POP_TOP POP_TOP designator POP_TOP
+                stmts_opt _jump COME_FROM
+                POP_TOP
+        except  ::=  POP_TOP POP_TOP POP_TOP
+                stmts_opt _jump
+
+        tryfinallystmt ::= SETUP_FINALLY stmts_opt
+                POP_BLOCK LOAD_CONST
+                COME_FROM stmts_opt END_FINALLY
+
+        whilestmt ::= SETUP_LOOP
+                expr JUMP_IF_FALSE POP_TOP
+                stmts_opt JUMP_ABSOLUTE
+                COME_FROM POP_TOP POP_BLOCK COME_FROM
+        while1stmt ::= SETUP_LOOP
+                JUMP_FORWARD JUMP_IF_FALSE POP_TOP COME_FROM
+                stmts_opt JUMP_ABSOLUTE
+                COME_FROM POP_TOP POP_BLOCK COME_FROM
+        whileelsestmt ::= SETUP_LOOP
+                          expr JUMP_IF_FALSE POP_TOP
+                stmts_opt JUMP_ABSOLUTE
+                COME_FROM POP_TOP POP_BLOCK
+                stmts COME_FROM
+        while1elsestmt ::= SETUP_LOOP
+                          JUMP_FORWARD JUMP_IF_FALSE POP_TOP COME_FROM
+                stmts_opt JUMP_ABSOLUTE
+                COME_FROM POP_TOP POP_BLOCK
+                stmts COME_FROM
+
+        _for ::= GET_ITER FOR_ITER
+        _for ::= LOAD_CONST FOR_LOOP
+
+        forstmt ::= SETUP_LOOP expr _for designator
+                stmts_opt JUMP_ABSOLUTE
+                COME_FROM POP_BLOCK COME_FROM
+        forelsestmt ::= SETUP_LOOP expr _for designator
+                stmts_opt JUMP_ABSOLUTE
+                COME_FROM POP_BLOCK stmts COME_FROM
+
+        stmt ::= ifforstmt
+        ifforstmt ::= expr condjmp SETUP_LOOP expr _for designator
+        stmts_opt JUMP_ABSOLUTE
+        COME_FROM POP_BLOCK _jump COME_FROM
+        POP_TOP COME_FROM COME_FROM
+
+        '''
+
+    def p_expr(self, args):
+        '''
+        expr ::= load_closure mklambda
+        expr ::= mklambda
+        expr ::= SET_LINENO
+        expr ::= LOAD_FAST
+        expr ::= LOAD_NAME
+        expr ::= LOAD_CONST
+        expr ::= LOAD_GLOBAL
+        expr ::= LOAD_DEREF
+        expr ::= LOAD_LOCALS
+        expr ::= expr LOAD_ATTR
+        expr ::= binary_expr
+        expr ::= build_list
+        
+        binary_expr ::= expr expr binary_op
+        binary_op ::= BINARY_ADD
+        binary_op ::= BINARY_SUBTRACT
+        binary_op ::= BINARY_MULTIPLY
+        binary_op ::= BINARY_DIVIDE
+        binary_op ::= BINARY_TRUE_DIVIDE
+        binary_op ::= BINARY_FLOOR_DIVIDE
+        binary_op ::= BINARY_MODULO
+        binary_op ::= BINARY_LSHIFT
+        binary_op ::= BINARY_RSHIFT
+        binary_op ::= BINARY_AND
+        binary_op ::= BINARY_OR
+        binary_op ::= BINARY_XOR
+        binary_op ::= BINARY_POWER
+
+        expr ::= binary_subscr
+        binary_subscr ::= expr expr BINARY_SUBSCR
+        expr ::= expr expr DUP_TOPX_2 BINARY_SUBSCR
+        expr ::= cmp
+        expr ::= expr UNARY_POSITIVE
+        expr ::= expr UNARY_NEGATIVE
+        expr ::= expr UNARY_CONVERT
+        expr ::= expr UNARY_INVERT
+        expr ::= expr UNARY_NOT
+        expr ::= mapexpr
+        expr ::= expr SLICE+0
+        expr ::= expr expr SLICE+1
+        expr ::= expr expr SLICE+2
+        expr ::= expr expr expr SLICE+3
+        expr ::= expr DUP_TOP SLICE+0
+        expr ::= expr expr DUP_TOPX_2 SLICE+1
+        expr ::= expr expr DUP_TOPX_2 SLICE+2
+        expr ::= expr expr expr DUP_TOPX_3 SLICE+3
+        expr ::= and
+        expr ::= and2
+        expr ::= or
+        or   ::= expr JUMP_IF_TRUE  POP_TOP expr COME_FROM
+        and  ::= expr JUMP_IF_FALSE POP_TOP expr COME_FROM
+        and2 ::= _jump JUMP_IF_FALSE POP_TOP COME_FROM expr COME_FROM
+        
+        _25_bexp ::= expr
+        _25_bexp ::= _25_and
+        _25_bexp ::= _25_or
+        _25_bexp_end ::= _25_and_end
+        _25_bexp_end ::= _25_or_end
+        
+        _25_and ::= expr JUMP_IF_FALSE POP_TOP expr
+        _25_and ::= _25_bexp JUMP_IF_FALSE _25_come_froms _25_bexp
+        _25_and ::= _25_bexp JUMP_IF_FALSE _25_come_froms _25_bexp_end
+        _25_and_end ::= expr JUMP_IF_FALSE
+        
+        _25_or ::= expr JUMP_IF_TRUE POP_TOP expr
+        _25_or ::= _25_bexp JUMP_IF_TRUE _25_come_froms _25_bexp
+        _25_or ::= _25_bexp JUMP_IF_TRUE _25_come_froms _25_bexp_end
+        _25_or_end ::= expr JUMP_IF_TRUE
+
+        cmp ::= cmp_list
+        cmp ::= compare
+        compare ::= expr expr COMPARE_OP
+        cmp_list ::= expr cmp_list1 ROT_TWO POP_TOP
+                COME_FROM
+        cmp_list1 ::= expr DUP_TOP ROT_THREE
+                COMPARE_OP JUMP_IF_FALSE POP_TOP
+                cmp_list1 COME_FROM
+        cmp_list1 ::= expr DUP_TOP ROT_THREE
+                COMPARE_OP JUMP_IF_FALSE POP_TOP
+                cmp_list2 COME_FROM
+        cmp_list2 ::= expr COMPARE_OP JUMP_FORWARD
+        mapexpr ::= BUILD_MAP kvlist
+
+        kvlist ::= kvlist kv
+        kvlist ::= kvlist kv2
+        kvlist ::=
+
+        kv ::= DUP_TOP expr ROT_TWO expr STORE_SUBSCR
+        kv2 ::= DUP_TOP expr expr ROT_THREE STORE_SUBSCR
+
+        exprlist ::= exprlist expr
+        exprlist ::= expr
+
+        nullexprlist ::=
+        '''
+
+    def nonterminal(self, nt, args):
+        collect = ('stmts', 'exprlist', 'kvlist')
+
+        if nt in collect and len(args) > 1:
+            #
+            #  Collect iterated thingies together.
+            #
+            rv = args[0]
+            rv.append(args[1])
+        else:
+            rv = GenericASTBuilder.nonterminal(self, nt, args)
+        return rv
+
+    def __ambiguity(self, children):
+        # only for debugging! to be removed hG/2000-10-15
+        print children
+        return GenericASTBuilder.ambiguity(self, children)
+
+    def resolve(self, list):
+        if len(list) == 2 and 'funcdef' in list and 'assign' in list:
+            return 'funcdef'
+        #print >> sys.stderr, 'resolve', str(list)
+        return GenericASTBuilder.resolve(self, list)
+
+nop = lambda self, args: None
+
+p = Parser()
+
+def parse(tokens, customize):
+    #
+    #  Special handling for opcodes that take a variable number
+    #  of arguments -- we add a new rule for each:
+    #
+    #    expr ::= {expr}^n BUILD_LIST_n
+    #    expr ::= {expr}^n BUILD_TUPLE_n
+    #    expr ::= {expr}^n BUILD_SLICE_n
+    #    unpack_list ::= UNPACK_LIST {expr}^n
+    #    unpack ::= UNPACK_TUPLE {expr}^n
+    #    unpack ::= UNPACK_SEQEUENE {expr}^n
+    #    mkfunc ::= {expr}^n LOAD_CONST MAKE_FUNCTION_n
+    #    mkfunc ::= {expr}^n load_closure LOAD_CONST MAKE_FUNCTION_n
+    #    expr ::= expr {expr}^n CALL_FUNCTION_n
+    #    expr ::= expr {expr}^n CALL_FUNCTION_VAR_n POP_TOP
+    #    expr ::= expr {expr}^n CALL_FUNCTION_VAR_KW_n POP_TOP
+    #    expr ::= expr {expr}^n CALL_FUNCTION_KW_n POP_TOP
+    #
+    global p
+    for k, v in customize.items():
+        # avoid adding the same rule twice to this parser
+        if p.customized.has_key(k):
+            continue
+        p.customized[k] = None
+
+        #nop = lambda self, args: None
+        op = k[:string.rfind(k, '_')]
+        if op in ('BUILD_LIST', 'BUILD_TUPLE'):
+            rule = 'build_list ::= ' + 'expr '*v + k
+        elif op == 'BUILD_SLICE':
+            rule = 'expr ::= ' + 'expr '*v + k
+        elif op in ('UNPACK_TUPLE', 'UNPACK_SEQUENCE'):
+            rule = 'unpack ::= ' + k + ' designator'*v
+        elif op == 'UNPACK_LIST':
+            rule = 'unpack_list ::= ' + k + ' designator'*v
+        elif op == 'DUP_TOPX':
+            # no need to add a rule
+            continue
+            #rule = 'dup_topx ::= ' + 'expr '*v + k
+        elif op == 'MAKE_FUNCTION':
+            p.addRule('mklambda ::= %s LOAD_LAMBDA %s' %
+                  ('expr '*v, k), nop)
+            rule = 'mkfunc ::= %s LOAD_CONST %s' % ('expr '*v, k)
+        elif op == 'MAKE_CLOSURE':
+            p.addRule('mklambda ::= %s load_closure LOAD_LAMBDA %s' %
+                  ('expr '*v, k), nop)
+            rule = 'mkfunc ::= %s load_closure LOAD_CONST %s' % ('expr '*v, k)
+            rule = 'mkfunc ::= %s closure_list LOAD_CONST %s' % ('expr '*v, k)
+        elif op in ('CALL_FUNCTION', 'CALL_FUNCTION_VAR',
+                'CALL_FUNCTION_VAR_KW', 'CALL_FUNCTION_KW'):
+            na = (v & 0xff)           # positional parameters
+            nk = (v >> 8) & 0xff      # keyword parameters
+            # number of apply equiv arguments:
+            nak = ( len(op)-len('CALL_FUNCTION') ) / 3
+            rule = 'expr ::= expr ' + 'expr '*na + 'kwarg '*nk \
+                   + 'expr ' * nak + k
+        # CE - Hack for >= 2.5
+        #      Now all values loaded via LOAD_CLOSURE are packed into
+        #      a tuple before calling MAKE_CLOSURE.
+        elif op == 'CLOSURE_LIST':
+            rule = 'closure_list ::= load_closure %s' % k
+        else:
+            raise 'unknown customize token %s' % k
+        p.addRule(rule, nop)
+    ast = p.parse(tokens)
+    #p.cleanup()
+    return ast

File uncompyle/Scanner.py

+#  Copyright (c) 1999 John Aycock
+#  Copyright (c) 2000-2002 by hartmut Goebel <hartmut@goebel.noris.de>
+#  Copyright (c) 2005 by Dan Pascu <dan@windowmaker.org>
+#
+#  See main module for license.
+#
+
+__all__ = ['Token', 'Scanner', 'getscanner']
+
+import types
+
+class Token:
+    """
+    Class representing a byte-code token.
+    
+    A byte-code token is equivalent to the contents of one line
+    as output by dis.dis().
+    """
+    def __init__(self, type, attr=None, pattr=None, offset=-1):
+        self.type = intern(type)
+        self.attr = attr
+        self.pattr = pattr
+        self.offset = offset
+        
+    def __cmp__(self, o):
+        if isinstance(o, Token):
+            # both are tokens: compare type and pattr 
+            return cmp(self.type, o.type) or cmp(self.pattr, o.pattr)
+        else:
+            return cmp(self.type, o)
+
+    def __repr__(self):		return str(self.type)
+    def __str__(self):
+        pattr = self.pattr or ''
+        return '%s\t%-17s %r' % (self.offset, self.type, pattr)
+    def __hash__(self):		return hash(self.type)
+    def __getitem__(self, i):	raise IndexError
+
+
+class Code:
+    """
+    Class for representing code-objects.
+
+    This is similar to the original code object, but additionally
+    the diassembled code is stored in the attribute '_tokens'.
+    """
+    def __init__(self, co, scanner):
+        for i in dir(co):
+            if i.startswith('co_'):
+                setattr(self, i, getattr(co, i))
+        self._tokens, self._customize = scanner.disassemble(co)
+
+class Scanner:
+    def __init__(self, version):
+        self.__version = version
+
+        from sys import version_info
+        self.__pyversion = float('%d.%d' % version_info[0:2])
+
+        # use module 'dis' for this version
+        import dis_files
+        self.dis = dis_files.by_version[version]
+        self.resetTokenClass()
+
+        dis = self.dis
+        self.JUMP_OPs = map(lambda op: dis.opname[op],
+                            dis.hasjrel + dis.hasjabs)
+
+        # setup an opmap if we don't have one
+        try:
+            dis.opmap
+        except:
+            opmap = {}
+            opname = dis.opname
+            for i in range(len(opname)):
+                opmap[opname[i]] = i
+            dis.opmap = opmap
+        copmap = {}
+        for i in range(len(dis.cmp_op)):
+            copmap[dis.cmp_op[i]] = i
+        dis.copmap = copmap
+
+    def setShowAsm(self, showasm, out=None):
+        self.showasm = showasm
+        self.out = out
+            
+    def setTokenClass(self, tokenClass):
+        assert type(tokenClass) == types.ClassType
+        self.Token = tokenClass
+        
+    def resetTokenClass(self):
+        self.setTokenClass(Token)
+        
+    def disassemble(self, co):
+        """
+        Disassemble a code object, returning a list of 'Token'.
+
+        The main part of this procedure is modelled after
+        dis.disassemble().
+        """
+        rv = []
+        customize = {}
+        dis = self.dis # shortcut
+        Token = self.Token # shortcut
+
+        code = co.co_code
+        cf = self.find_jump_targets(code)
+        n = len(code)
+        i = 0
+        extended_arg = 0
+        free = None
+        while i < n:
+            offset = i
+            if cf.has_key(offset):
+                for j in range(cf[offset]):
+                    rv.append(Token('COME_FROM',
+                                    offset="%s_%d" % (offset, j) ))
+
+            c = code[i]
+            op = ord(c)
+            opname = dis.opname[op]
+            i += 1
+            oparg = None; pattr = None
+            if op >= dis.HAVE_ARGUMENT:
+                oparg = ord(code[i]) + ord(code[i+1]) * 256 + extended_arg
+                extended_arg = 0
+                i += 2
+                if op == dis.EXTENDED_ARG:
+                    extended_arg = oparg * 65536L
+                if op in dis.hasconst:
+                    const = co.co_consts[oparg]
+                    if type(const) == types.CodeType:
+                        oparg = const
+                        if const.co_name == '<lambda>':
+                            assert opname == 'LOAD_CONST'
+                            opname = 'LOAD_LAMBDA'
+                        # verify uses 'pattr' for comparism, since 'attr'
+                        # now holds Code(const) and thus can not be used
+                        # for comparism (todo: think about changing this)
+                        #pattr = 'code_object @ 0x%x %s->%s' %\
+                        #	(id(const), const.co_filename, const.co_name)
+                        pattr = '<code_object ' + const.co_name + '>'
+                    else:
+                        pattr = const
+                elif op in dis.hasname:
+                    pattr = co.co_names[oparg]
+                elif op in dis.hasjrel:
+                    pattr = repr(i + oparg)
+                elif op in dis.hasjabs:
+                    pattr = repr(oparg)
+                elif op in dis.haslocal:
+                    pattr = co.co_varnames[oparg]
+                elif op in dis.hascompare:
+                    pattr = dis.cmp_op[oparg]
+                elif op in dis.hasfree:
+                    if free is None:
+                        free = co.co_cellvars + co.co_freevars
+                    pattr = free[oparg]
+
+            if opname == 'SET_LINENO':
+                continue
+            elif opname in ('BUILD_LIST', 'BUILD_TUPLE', 'BUILD_SLICE',
+                            'UNPACK_LIST', 'UNPACK_TUPLE', 'UNPACK_SEQUENCE',
+                            'MAKE_FUNCTION', 'CALL_FUNCTION', 'MAKE_CLOSURE',
+                            'CALL_FUNCTION_VAR', 'CALL_FUNCTION_KW',
+                            'CALL_FUNCTION_VAR_KW', 'DUP_TOPX',
+                            ):
+                # CE - Hack for >= 2.5
+                #      Now all values loaded via LOAD_CLOSURE are packed into
+                #      a tuple before calling MAKE_CLOSURE.
+                if opname == 'BUILD_TUPLE' and \
+                   dis.opname[ord(code[offset-3])] == 'LOAD_CLOSURE':
+                    opname = 'CLOSURE_LIST_%d' % oparg
+                else:
+                    opname = '%s_%d' % (opname, oparg)
+                customize[opname] = oparg
+
+            rv.append(Token(opname, oparg, pattr, offset))
+
+        if self.showasm:
+            out = self.out # shortcut
+            for t in rv:
+                print >>out, t
+            print >>out
+
+        return rv, customize
+
+    def __get_target(self, code, pos, op=None):
+        if op is None:
+            op = ord(code[pos])
+        target = ord(code[pos+1]) + ord(code[pos+2]) * 256
+        if op in self.dis.hasjrel:
+            target += pos + 3
+        return target
+
+    def __first_instr(self, code, start, end, instr, target=None, exact=True):
+        """
+        Find the first <instr> in the block from start to end.
+        <instr> is any python bytecode instruction or a list of opcodes
+        If <instr> is an opcode with a target (like a jump), a target
+        destination can be specified which must match precisely if exact
+        is True, or if exact is False, the instruction which has a target
+        closest to <target> will be returned.
+
+        Return index to it or None if not found.
+        """
+
+        assert(start>=0 and end<len(code))
+
+        HAVE_ARGUMENT = self.dis.HAVE_ARGUMENT
+
+        try:    instr[0]
+        except: instr = [instr]
+
+        pos = None
+        distance = len(code)
+        i = start
+        while i < end:
+            op = ord(code[i])
+            if op in instr:
+                if target is None:
+                    return i
+                dest = self.__get_target(code, i, op)
+                if dest == target:
+                    return i
+                elif not exact:
+                    _distance = abs(target - dest)
+                    if _distance < distance:
+                        distance = _distance
+                        pos = i
+            if op < HAVE_ARGUMENT:
+                i += 1
+            else:
+                i += 3
+        return pos
+
+    def __last_instr(self, code, start, end, instr, target=None, exact=True):
+        """
+        Find the last <instr> in the block from start to end.
+        <instr> is any python bytecode instruction or a list of opcodes
+        If <instr> is an opcode with a target (like a jump), a target
+        destination can be specified which must match precisely if exact
+        is True, or if exact is False, the instruction which has a target
+        closest to <target> will be returned.
+
+        Return index to it or None if not found.
+        """
+
+        assert(start>=0 and end<len(code))
+
+        HAVE_ARGUMENT = self.dis.HAVE_ARGUMENT
+
+        try:    instr[0]
+        except: instr = [instr]
+
+        pos = None
+        distance = len(code)
+        i = start
+        while i < end:
+            op = ord(code[i])
+            if op in instr:
+                if target is None:
+                    pos = i
+                else:
+                    dest = self.__get_target(code, i, op)
+                    if dest == target:
+                        distance = 0
+                        pos = i
+                    elif not exact:
+                        _distance = abs(target - dest)
+                        if _distance <= distance:
+                            distance = _distance
+                            pos = i
+            if op < HAVE_ARGUMENT:
+                i += 1
+            else:
+                i += 3
+        return pos
+
+    def __all_instr(self, code, start, end, instr, target=None):
+        """
+        Find all <instr> in the block from start to end.
+        <instr> is any python bytecode instruction or a list of opcodes
+        If <instr> is an opcode with a target (like a jump), a target
+        destination can be specified which must match precisely.
+
+        Return a list with indexes to them or [] if none found.
+        """
+
+        assert(start>=0 and end<len(code))
+
+        HAVE_ARGUMENT = self.dis.HAVE_ARGUMENT
+
+        try:    instr[0]
+        except: instr = [instr]
+
+        result = []
+        i = start
+        while i < end:
+            op = ord(code[i])
+            if op in instr:
+                if target is None:
+                    result.append(i)
+                elif target == self.__get_target(code, i, op):
+                    result.append(i)
+            if op < HAVE_ARGUMENT:
+                i += 1
+            else:
+                i += 3
+        return result
+
+    def __next_except_jump(self, code, start, end, target):
+        """
+        Return the next jump that was generated by an except SomeException:
+        construct in a try...except...else clause or None if not found.
+        """
+        HAVE_ARGUMENT = self.dis.HAVE_ARGUMENT
+        JUMP_FORWARD  = self.dis.opmap['JUMP_FORWARD']
+        JUMP_ABSOLUTE = self.dis.opmap['JUMP_ABSOLUTE']
+        END_FINALLY   = self.dis.opmap['END_FINALLY']
+        POP_TOP       = self.dis.opmap['POP_TOP']
+        DUP_TOP       = self.dis.opmap['DUP_TOP']
+        try:    SET_LINENO = self.dis.opmap['SET_LINENO']
+        except: SET_LINENO = None
+
+        lookup = [JUMP_ABSOLUTE, JUMP_FORWARD]
+        while start < end:
+            jmp = self.__first_instr(code, start, end, lookup, target)
+            if jmp is None:
+                return None
+            if jmp == end-3:
+                return jmp
+            after = jmp + 3
+            ops = [None, None, None, None]
+            opp = [0, 0, 0, 0]
+            pos = 0
+            x = jmp+3
+            while x <= end and pos < 4:
+                op = ord(code[x])
+                if op == SET_LINENO:
+                    x += 3
+                    continue
+                elif op >= HAVE_ARGUMENT:
+                    break
+                ops[pos] = op
+                opp[pos] = x
+                pos += 1
+                x += 1
+            if ops[0] == POP_TOP and ops[1] == END_FINALLY and opp[1] == end:
+                return jmp
+            if ops[0] == POP_TOP and ops[1] == DUP_TOP:
+                return jmp
+            if ops[0] == ops[1] == ops[2] == ops[3] == POP_TOP:
+                return jmp
+            start = jmp + 3
+        return None
+
+    def __list_comprehension(self, code, pos, op=None):
+        """
+        Determine if there is a list comprehension structure starting at pos
+        """
+        BUILD_LIST = self.dis.opmap['BUILD_LIST']
+        DUP_TOP    = self.dis.opmap['DUP_TOP']
+        LOAD_ATTR  = self.dis.opmap['LOAD_ATTR']
+        if op is None:
+            op = ord(code[pos])
+        if op != BUILD_LIST:
+            return 0
+        try:
+            elems = ord(code[pos+1]) + ord(code[pos+2])*256
+            codes = (op, elems, ord(code[pos+3]), ord(code[pos+4]))
+        except IndexError:
+            return 0
+        return (codes==(BUILD_LIST, 0, DUP_TOP, LOAD_ATTR))
+
+    def __ignore_if(self, code, pos):
+        """
+        Return true if this 'if' is to be ignored.
+        """
+        POP_TOP      = self.dis.opmap['POP_TOP']
+        COMPARE_OP   = self.dis.opmap['COMPARE_OP']
+        EXCEPT_MATCH = self.dis.copmap['exception match']
+
+        ## If that was added by a while loop
+        if pos in self.__ignored_ifs:
+            return 1
+
+        # Check if we can test only for POP_TOP for this -Dan
+        # Maybe need to be done as above (skip SET_LINENO's)
+        if (ord(code[pos-3])==COMPARE_OP and
+            (ord(code[pos-2]) + ord(code[pos-1])*256)==EXCEPT_MATCH and
+            ord(code[pos+3])==POP_TOP and
+            ord(code[pos+4])==POP_TOP and
+            ord(code[pos+5])==POP_TOP and
+            ord(code[pos+6])==POP_TOP):
+            return 1 ## Exception match
+        return 0
+
+    def __fix_parent(self, code, target, parent):
+        """Fix parent boundaries if needed"""
+        JUMP_ABSOLUTE = self.dis.opmap['JUMP_ABSOLUTE']
+        start = parent['start']
+        end = parent['end']
+        ## Map the second start point for 'while 1:' in python 2.3+ to start
+        try:    target = self.__while1[target]
+        except: pass
+        if target >= start or end-start < 3 or target not in self.__loops:
+            return
+        if ord(code[end-3])==JUMP_ABSOLUTE:
+            cont_target = self.__get_target(code, end-3, JUMP_ABSOLUTE)
+            if target == cont_target:
+                parent['end'] = end-3
+
+    def __restrict_to_parent(self, target, parent):
+        """Restrict pos to parent boundaries."""
+        if not (parent['start'] < target < parent['end']):
+            target = parent['end']
+        return target
+
+    def __detect_structure(self, code, pos, op=None):
+        """
+        Detect structures and their boundaries to fix optimizied jumps
+        in python2.3+
+        """
+
+        # TODO: check the struct boundaries more precisely -Dan
+
+        SETUP_LOOP    = self.dis.opmap['SETUP_LOOP']
+        BUILD_LIST    = self.dis.opmap['BUILD_LIST']
+        FOR_ITER      = self.dis.opmap['FOR_ITER']
+        GET_ITER      = self.dis.opmap['GET_ITER']
+        SETUP_EXCEPT  = self.dis.opmap['SETUP_EXCEPT']
+        JUMP_FORWARD  = self.dis.opmap['JUMP_FORWARD']
+        JUMP_ABSOLUTE = self.dis.opmap['JUMP_ABSOLUTE']
+        JUMP_IF_FALSE = self.dis.opmap['JUMP_IF_FALSE']
+        JUMP_IF_TRUE  = self.dis.opmap['JUMP_IF_TRUE']
+        END_FINALLY   = self.dis.opmap['END_FINALLY']
+        POP_TOP       = self.dis.opmap['POP_TOP']
+        try:    SET_LINENO = self.dis.opmap['SET_LINENO']
+        except: SET_LINENO = None
+
+        # Ev remove this test and make op a mandatory argument -Dan
+        if op is None:
+            op = ord(code[pos])
+
+        ## Detect parent structure
+        parent = self.__structs[0]
+        start  = parent['start']
+        end    = parent['end']
+        for s in self.__structs:
+            _start = s['start']
+            _end   = s['end']
+            if (_start <= pos < _end) and (_start >= start and _end < end):
+                start  = _start
+                end    = _end
+                parent = s
+
+        ## We need to know how many new structures were added in this run
+        origStructCount = len(self.__structs)
+
+        if op == SETUP_LOOP:
+            start = pos+3
+            if ord(code[start])==JUMP_FORWARD:
+                ## This is a while 1 (has a particular structure)
+                start = self.__get_target(code, start, JUMP_FORWARD)
+                start = self.__restrict_to_parent(start, parent)
+                self.__while1[pos+3] = start ## map between the 2 start points
+            target = self.__get_target(code, pos, op)
+            end    = self.__restrict_to_parent(target, parent)
+            if target != end:
+                self.__fixed_jumps[pos] = end
+            jump_back = self.__last_instr(code, start, end, JUMP_ABSOLUTE,
+                                          start, False)
+            assert(jump_back is not None)
+            target = self.__get_target(code, jump_back, JUMP_ABSOLUTE)
+            i = target
+            while i < jump_back and ord(code[i])==SET_LINENO:
+                i += 3
+            if ord(code[i]) in (FOR_ITER, GET_ITER):
+                loop_type = 'for'
+            else:
+                loop_type = 'while'
+                lookup = [JUMP_IF_FALSE, JUMP_IF_TRUE]
+                test = self.__first_instr(code, pos+3, jump_back, lookup)
+                assert(test is not None)
+                self.__ignored_ifs.append(test)
+            self.__loops.append(target)
+            self.__structs.append({'type': loop_type + '-loop',
+                                   'start': target,
+                                   'end':   jump_back})
+            self.__structs.append({'type': loop_type + '-else',
+                                   'start': jump_back+3,
+                                   'end':   end})
+        elif self.__list_comprehension(code, pos, op):
+            get_iter = self.__first_instr(code, pos+7, end, GET_ITER)
+            for_iter = self.__first_instr(code, get_iter, end, FOR_ITER)
+            assert(get_iter is not None and for_iter is not None)
+            start  = get_iter+1
+            target = self.__get_target(code, for_iter, FOR_ITER)
+            end    = self.__restrict_to_parent(target, parent)
+            jump_back = self.__last_instr(code, start, end, JUMP_ABSOLUTE,
+                                          start, False)
+            assert(jump_back is not None)
+            target = self.__get_target(code, jump_back, JUMP_ABSOLUTE)
+            start = self.__restrict_to_parent(target, parent)
+            self.__structs.append({'type': 'list-comprehension',
+                                   'start': start,
+                                   'end':   jump_back})
+        elif op == SETUP_EXCEPT:
+            start  = pos+3
+            target = self.__get_target(code, pos, op)
+            end    = self.__restrict_to_parent(target, parent)
+            if target != end:
+                self.__fixed_jumps[pos] = end
+            ## Add the try block
+            self.__structs.append({'type':  'try',
+                                   'start': start,
+                                   'end':   end-4})
+            ## Now isolate the except and else blocks
+            start  = end
+            target = self.__get_target(code, start-3)
+            self.__fix_parent(code, target, parent)
+            end    = self.__restrict_to_parent(target, parent)
+            if target != end:
+                self.__fixed_jumps[start-3] = end
+
+            end_finally = self.__last_instr(code, start, end, END_FINALLY)
+            assert(end_finally is not None)
+            lookup = [JUMP_ABSOLUTE, JUMP_FORWARD]
+            jump_end = self.__last_instr(code, start, end, lookup)
+            assert(jump_end is not None)
+
+            target = self.__get_target(code, jump_end)
+            end = self.__restrict_to_parent(target, parent)
+            if target != end:
+                self.__fixed_jumps[jump_end] = end
+
+            ## Add the try-else block
+            self.__structs.append({'type':  'try-else',
+                                   'start': end_finally+1,
+                                   'end':   end})
+            ## Add the except blocks
+            i = start
+            while i < end_finally:
+                jmp = self.__next_except_jump(code, i, end_finally, target)
+                if jmp is None:
+                    break
+                self.__structs.append({'type':  'except',
+                                       'start': i,
+                                       'end':   jmp})
+                if target != end:
+                    self.__fixed_jumps[jmp] = end
+                i = jmp+3
+        elif op == JUMP_ABSOLUTE:
+            ## detect if we have a 'foo and bar and baz...' structure
+            ## that was optimized (thus the presence of JUMP_ABSOLUTE)
+            if pos in self.__fixed_jumps:
+                return ## Already marked
+            if parent['end'] - pos < 7:
+                return
+            next = (ord(code[pos+3]), ord(code[pos+6]))
+            if next != (JUMP_IF_FALSE, POP_TOP):
+                return
+
+            end = self.__get_target(code, pos+3)
+            ifs = self.__all_instr(code, pos, end, JUMP_IF_FALSE, end)
+
+            ## Test if all JUMP_IF_FALSE we have found belong to the
+            ## structure (may not be needed but it doesn't hurt)
+            count = len(ifs)
+            if count < 2:
+                return
+            for jif in ifs[1:]:
+                before = ord(code[jif-3])
+                after  = ord(code[jif+3])
+                if (before not in (JUMP_FORWARD, JUMP_ABSOLUTE) or
+                    after != POP_TOP):
+                    return
+
+            ## All tests passed. Perform fixes
+            self.__ignored_ifs.extend(ifs)
+            for i in range(count-1):
+                self.__fixed_jumps[ifs[i]-3] = ifs[i+1]-3
+        elif op in (JUMP_IF_FALSE, JUMP_IF_TRUE):
+            if self.__ignore_if(code, pos):
+                return
+            start  = pos+4 ## JUMP_IF_FALSE/TRUE + POP_TOP
+            target = self.__get_target(code, pos, op)
+            if parent['start'] <= target < parent['end']:
+                if ord(code[target-3]) in (JUMP_ABSOLUTE, JUMP_FORWARD):
+                    if_end = self.__get_target(code, target-3)
+                    self.__fix_parent(code, if_end, parent)
+                    end = self.__restrict_to_parent(if_end, parent)
+                    if if_end != end:
+                        self.__fixed_jumps[target-3] = end
+                    self.__structs.append({'type':  'if-then',
+                                           'start': start,
+                                           'end':   target-3})
+                    self.__structs.append({'type':  'if-else',
+                                           'start': target+1,
+                                           'end':   end})
+
+    def find_jump_targets(self, code):
+        """
+        Detect all offsets in a byte code which are jump targets.
+
+        Return the list of offsets.
+
+        This procedure is modelled after dis.findlables(), but here
+        for each target the number of jumps are counted.
+        """
+        HAVE_ARGUMENT = self.dis.HAVE_ARGUMENT
+
+        hasjrel = self.dis.hasjrel
+        hasjabs = self.dis.hasjabs
+
+        needFixing = (self.__pyversion >= 2.3)
+
+        n = len(code)
+        self.__structs = [{'type':  'root',
+                           'start': 0,
+                           'end':   n-1}]
+        self.__loops = []  ## All loop entry points
+        self.__while1 = {} ## 'while 1:' in python 2.3+ has another start point
+        self.__fixed_jumps = {} ## Map fixed jumps to their real destination
+        self.__ignored_ifs = [] ## JUMP_IF_XXXX's we should ignore
+
+        targets = {}
+        i = 0
+        while i < n:
+            op = ord(code[i])
+
+            if needFixing:
+                ## Determine structures and fix jumps for 2.3+
+                self.__detect_structure(code, i, op)
+
+            if op >= HAVE_ARGUMENT:
+                label = self.__fixed_jumps.get(i)
+                if label is None:
+                    oparg = ord(code[i+1]) + ord(code[i+2]) * 256
+                    if op in hasjrel:
+                        label = i + 3 + oparg
+                    elif op in hasjabs:
+                        # todo: absolute jumps
+                        pass
+                if label is not None:
+                    targets[label] = targets.get(label, 0) + 1
+                i += 3
+            else:
+                i += 1
+        return targets
+
+
+__scanners = {}
+
+def getscanner(version):
+    if not __scanners.has_key(version):
+        __scanners[version] = Scanner(version)
+    return __scanners[version]

File uncompyle/Walker.py

+#  Copyright (c) 1999 John Aycock
+#  Copyright (c) 2000-2002 by hartmut Goebel <hartmut@goebel.noris.de>
+#  Copyright (c) 2005 by Dan Pascu <dan@windowmaker.org>
+#
+#  See main module for license.
+#
+#
+#  Decompilation (walking AST)
+#
+#  All table-driven.  Step 1 determines a table (T) and a path to a
+#  table key (K) from the node type (N) (other nodes are shown as O):
+#
+#         N                  N               N&K
+#     / | ... \          / | ... \        / | ... \
+#    O  O      O        O  O      K      O  O      O
+#              |
+#              K
+#
+#  MAP_R0 (TABLE_R0)  MAP_R (TABLE_R)  MAP_DIRECT (TABLE_DIRECT)
+#
+#  The default is a direct mapping.  The key K is then extracted from the
+#  subtree and used to find a table entry T[K], if any.  The result is a
+#  format string and arguments (a la printf()) for the formatting engine.
+#  Escapes in the format string are:
+#
+#	%c	evaluate N[A] recursively*
+#	%C	evaluate N[A[0]]..N[A[1]] recursively, separate by A[2]*
+#	%,	print ',' if last %C only printed one item (for tuples--unused)
+#	%|	tab to current indentation level
+#	%+	increase current indentation level
+#	%-	decrease current indentation level
+#	%{...}	evaluate ... in context of N
+#	%%	literal '%'
+#
+#  * indicates an argument (A) required.
+#
+#  The '%' may optionally be followed by a number (C) in square brackets, which
+#  makes the engine walk down to N[C] before evaluating the escape code.
+#
+
+import sys, re, cStringIO
+from types import ListType, TupleType, DictType, \
+     EllipsisType, IntType, CodeType
+
+from spark import GenericASTTraversal
+import Parser
+from Parser import AST
+from Scanner import Token, Code
+
+minint = -sys.maxint-1
+
+# Some ASTs used for comparing code fragments (like 'return None' at
+# the end of functions).
+
+RETURN_LOCALS = AST('stmt',
+		    [ AST('return_stmt',
+			  [ AST('expr', [ Token('LOAD_LOCALS') ]),
+			    Token('RETURN_VALUE')]) ])
+
+
+NONE = AST('expr', [ Token('LOAD_CONST', pattr=None) ] )
+
+RETURN_NONE = AST('stmt',
+		  [ AST('return_stmt',
+			[ NONE, Token('RETURN_VALUE')]) ])
+
+ASSIGN_DOC_STRING = lambda doc_string: \
+	AST('stmt',
+	    [ AST('assign',
+		  [ AST('expr', [ Token('LOAD_CONST', pattr=doc_string) ]),
+		    AST('designator', [ Token('STORE_NAME', pattr='__doc__')])
+		    ])])
+
+BUILD_TUPLE_0 = AST('expr',
+                    [ AST('build_list',
+                          [ Token('BUILD_TUPLE_0') ])])
+
+#TAB = '\t'			# as God intended
+TAB = ' ' *4   # is less spacy than "\t"
+INDENT_PER_LEVEL = ' ' # additional intent per pretty-print level
+
+TABLE_R = {
+    'build_tuple2':	( '%C', (0,-1,', ') ),
+    'POP_TOP':		( '%|%c\n', 0 ),
+    'STORE_ATTR':	( '%c.%[1]{pattr}', 0),
+#   'STORE_SUBSCR':	( '%c[%c]', 0, 1 ),
+    'STORE_SLICE+0':	( '%c[:]', 0 ),
+    'STORE_SLICE+1':	( '%c[%c:]', 0, 1 ),
+    'STORE_SLICE+2':	( '%c[:%c]', 0, 1 ),
+    'STORE_SLICE+3':	( '%c[%c:%c]', 0, 1, 2 ),
+    'JUMP_ABSOLUTE':	( '%|continue\n', ),
+    'DELETE_SLICE+0':	( '%|del %c[:]\n', 0 ),
+    'DELETE_SLICE+1':	( '%|del %c[%c:]\n', 0, 1 ),
+    'DELETE_SLICE+2':	( '%|del %c[:%c]\n', 0, 1 ),
+    'DELETE_SLICE+3':	( '%|del %c[%c:%c]\n', 0, 1, 2 ),
+    'DELETE_ATTR':	( '%|del %c.%[-1]{pattr}\n', 0 ),
+#   'EXEC_STMT':	( '%|exec %c in %[1]C\n', 0, (0,sys.maxint,', ') ),
+    'BINARY_SUBSCR':	( '%c[%c]', 0, 1), # required for augmented assign
+    'UNARY_POSITIVE':	( '+%c', 0 ),
+    'UNARY_NEGATIVE':	( '-%c', 0 ),
+    'UNARY_CONVERT':	( '`%c`', 0 ),
+    'UNARY_INVERT':	( '~%c', 0 ),
+    'UNARY_NOT':	( '(not %c)', 0 ),
+    'SLICE+0':		( '%c[:]', 0 ),
+    'SLICE+1':		( '%c[%c:]', 0, 1 ),
+    'SLICE+2':		( '%c[:%c]', 0, 1 ),
+    'SLICE+3':		( '%c[%c:%c]', 0, 1, 2 ),
+}
+TABLE_R0 = {
+#    'BUILD_LIST':	( '[%C]',      (0,-1,', ') ),
+#    'BUILD_TUPLE':	( '(%C)',      (0,-1,', ') ),
+#    'CALL_FUNCTION':	( '%c(%C)', 0, (1,-1,', ') ),
+}
+TABLE_DIRECT = {
+    'BINARY_ADD':	( '+' ,),
+    'BINARY_SUBTRACT':	( '-' ,),
+    'BINARY_MULTIPLY':	( '*' ,),
+    'BINARY_DIVIDE':	( '/' ,),
+    'BINARY_TRUE_DIVIDE':	( '/' ,),
+    'BINARY_FLOOR_DIVIDE':	( '//' ,),
+    'BINARY_MODULO':	( '%%',),
+    'BINARY_POWER':	( '**',),
+    'BINARY_LSHIFT':	( '<<',),
+    'BINARY_RSHIFT':	( '>>',),
+    'BINARY_AND':	( '&' ,),
+    'BINARY_OR':	( '|' ,),
+    'BINARY_XOR':	( '^' ,),
+    'INPLACE_ADD':	( '+=' ,),
+    'INPLACE_SUBTRACT':	( '-=' ,),
+    'INPLACE_MULTIPLY':	( '*=' ,),
+    'INPLACE_DIVIDE':	( '/=' ,),
+    'INPLACE_TRUE_DIVIDE':	( '/=' ,),
+    'INPLACE_FLOOR_DIVIDE':	( '//=' ,),
+    'INPLACE_MODULO':	( '%%=',),
+    'INPLACE_POWER':	( '**=',),
+    'INPLACE_LSHIFT':	( '<<=',),
+    'INPLACE_RSHIFT':	( '>>=',),
+    'INPLACE_AND':	( '&=' ,),
+    'INPLACE_OR':	( '|=' ,),
+    'INPLACE_XOR':	( '^=' ,),
+    'binary_expr':	( '(%c %c %c)', 0, -1, 1 ),
+
+    'IMPORT_FROM':	( '%{pattr}', ),
+    'LOAD_ATTR':	( '.%{pattr}', ),
+    'LOAD_FAST':	( '%{pattr}', ),
+    'LOAD_NAME':	( '%{pattr}', ),
+    'LOAD_GLOBAL':	( '%{pattr}', ),
+    'LOAD_DEREF':	( '%{pattr}', ),
+    'LOAD_LOCALS':	( 'locals()', ),
+#   'LOAD_CONST':	( '%{pattr}', ),	# handled by n_LOAD_CONST
+    'DELETE_FAST':	( '%|del %{pattr}\n', ),
+    'DELETE_NAME':	( '%|del %{pattr}\n', ),
+    'DELETE_GLOBAL':	( '%|del %{pattr}\n', ),
+    'delete_subscr':	( '%|del %c[%c]\n', 0, 1,),
+    'binary_subscr':	( '%c[%c]', 0, 1),
+    'store_subscr':	( '%c[%c]', 0, 1),
+    'STORE_FAST':	( '%{pattr}', ),
+    'STORE_NAME':	( '%{pattr}', ),
+    'STORE_GLOBAL':	( '%{pattr}', ),
+    'STORE_DEREF':	( '%{pattr}', ),
+    'unpack':		( '(%C,)', (1, sys.maxint, ', ') ),
+    'unpack_list':	( '[%C]', (1, sys.maxint, ', ') ),
+
+    #'list_compr':	( '[ %c ]', -2),	# handled by n_list_compr
+    'list_iter':	( '%c', 0),
+    'list_for':		( ' for %c in %c%c', 2, 0, 3 ),
+    'list_if':		( ' if %c%c', 0, 2 ),
+    'lc_body':		( '', ),	# ignore when recusing
+    
+    'assign':		( '%|%c = %c\n', -1, 0 ),
+    'augassign1':	( '%|%c %c %c\n', 0, 2, 1),
+    'augassign2':	( '%|%c%c %c %c\n', 0, 2, -3, -4),
+#   'dup_topx':		( '%c', 0),
+    'designList':	( '%c = %c', 0, -1 ),
+    'and':          	( '(%c and %c)', 0, 3 ),
+    'and2':          	( '%c', 4 ),
+    'or':           	( '(%c or %c)', 0, 3 ),
+    'compare':		( '(%c %[-1]{pattr} %c)', 0, 1 ),
+    'cmp_list':		( '%c %c', 0, 1),
+    'cmp_list1':	( '%[3]{pattr} %c %c', 0, -2),
+    'cmp_list2':	( '%[1]{pattr} %c', 0),
+#   'classdef': 	(), # handled by n_classdef()
+    'funcdef':  	( '\n%|def %c\n', -2), # -2 to handle closures
+    'kwarg':    	( '%[0]{pattr}=%c', 1),
+    'importstmt':	( '%|import %[0]{pattr}\n', ),
+    'importfrom':	( '%|from %[0]{pattr} import %c\n', 1 ),
+    'importlist':	( '%C', (0, sys.maxint, ', ') ),
+    'importstmt2':	( '%|import %c\n', 1),
+    'importstar2':	( '%|from %[1]{pattr} import *\n', ),
+    'importfrom2':	( '%|from %[1]{pattr} import %c\n', 2 ),
+    'importlist2':	( '%C', (0, sys.maxint, ', ') ),
+    
+    'assert':		( '%|assert %c\n' , 3 ),
+    'assert2':		( '%|assert %c, %c\n' , 3, -5 ),
+    'assert3':		( '%|assert %c\n' , 0 ),
+    'assert4':		( '%|assert %c, %c\n' , 0, -4 ),
+
+    'print_stmt':	( '%|print %c,\n', 0 ),
+    'print_stmt_nl':	( '%|print %[0]C\n', (0,1, None) ),
+    'print_nl_stmt':	( '%|print\n', ),
+    'print_to':		( '%|print >> %c, %c,\n', 0, 1 ),
+    'print_to_nl':	( '%|print >> %c, %c\n', 0, 1 ),
+    'print_nl_to':	( '%|print >> %c\n', 0 ),
+    'print_to_items':	( '%C', (0, 2, ', ') ),
+
+    'call_stmt':	( '%|%c\n', 0),
+    'break_stmt':	( '%|break\n', ),
+    'continue_stmt':	( '%|continue\n', ),
+    'raise_stmt':	( '%|raise %[0]C\n', (0,sys.maxint,', ') ),
+    'yield_stmt':	( '%|yield %c\n', 0),
+    'return_stmt':	( '%|return %c\n', 0),
+    'return_lambda':	( '%c', 0),
+
+    'ifstmt':		( '%|if %c:\n%+%c%-', 0, 2 ),
+    'ifelsestmt':	( '%|if %c:\n%+%c%-%|else:\n%+%c%-', 0, 2, -2 ),
+    'ifelifstmt':	( '%|if %c:\n%+%c%-%c', 0, 2, -2 ),
+    'elifelifstmt':	( '%|elif %c:\n%+%c%-%c', 0, 2, -2 ),
+    'elifstmt':		( '%|elif %c:\n%+%c%-', 0, 2 ),
+    'elifelsestmt':	( '%|elif %c:\n%+%c%-%|else:\n%+%c%-', 0, 2, -2 ),
+
+    'whilestmt':	( '%|while %c:\n%+%c%-\n', 1, 4 ),
+    'while1stmt':	( '%|while 1:\n%+%c%-\n', 5 ),
+    'whileelsestmt':	( '%|while %c:\n%+%c%-%|else:\n%+%c%-\n', 1, 4, -2 ),
+    'while1elsestmt':	( '%|while 1:\n%+%c%-%|else:\n%+%c%-\n', 5, -2 ),
+    'forstmt':		( '%|for %c in %c:\n%+%c%-\n', 3, 1, 4 ),
+    'forelsestmt':	(
+        '%|for %c in %c:\n%+%c%-%|else:\n%+%c%-\n', 3, 1, 4, -2
+     ),
+    'trystmt':		( '%|try:\n%+%c%-%c', 1, 5 ),
+    'except':		( '%|except:\n%+%c%-', 3 ),
+    'except_cond1':	( '%|except %c:\n%+%c%-', 1, 8 ),
+    'except_cond2':	( '%|except %c, %c:\n%+%c%-', 1, 6, 8 ),
+    'except_else':	( '%|else:\n%+%c%-', 2 ),
+    'tryfinallystmt':	( '%|try:\n%+%c%-\n%|finally:\n%+%c%-\n', 1, 5 ),
+    'passstmt':		( '%|pass\n', ),
+    'STORE_FAST':	( '%{pattr}', ),
+    'kv':		( '%c: %c', 3, 1 ),
+    'kv2':		( '%c: %c', 1, 2 ),
+    'mapexpr':		( '{%[1]C}', (0,sys.maxint,', ') ),
+    
+    ##
+    ## Python 2.5 Additions
+    ##
+    
+    # Import style for 2.5
+    '_25_importstmt': ( '%|import %c\n', 2),
+    '_25_importstar': ( '%|from %[2]{pattr} import *\n', ),
+    '_25_importfrom': ( '%|from %[2]{pattr} import %c\n', 3 ),
+    
+    # If statement fixes
+    # CE - These values should be correct by themselves, the main difference
+    #      is is the composition of and/or on the Parser side.
+    '_25_and':            ( '(%c and %c)', 0, 3 ),
+    '_25_and_end':        ( '%c', 0),
+    '_25_or':             ( '(%c or %c)', 0, 3),
+    '_25_or_end':         ( '%c', 0),
+    
+    # CE - These should also be fine as they are.
+    '_25_ifstmt':     ( '%|if %c:\n%+%c%-', 0, 1),
+    '_25_ifelsestmt': ( '%|if %c:\n%+%c%-%|else:\n%+%c%-', 0, 1, -2),
+    '_25_jumptarget_e': ( '%c', -1),
+    '_25_jumptarget_s': ( '%c', -1),
+}
+
+
+MAP_DIRECT = (TABLE_DIRECT, )
+MAP_R0 = (TABLE_R0, -1, 0)
+MAP_R = (TABLE_R, -1)
+
+MAP = {
+    'stmt':		MAP_R,
+    'del_stmt':		MAP_R,
+    'designator':	MAP_R,
+    'expr':		MAP_R,
+    'exprlist':		MAP_R0,
+}
+
+
+ASSIGN_TUPLE_PARAM = lambda param_name: \
+             AST('expr', [ Token('LOAD_FAST', pattr=param_name) ])
+
+escape = re.compile(r'''
+            (?P<prefix> [^%]* )
+            % ( \[ (?P<child> -? \d+ ) \] )?
+                ((?P<type> [^{] ) |
+                 ( [{] (?P<expr> [^}]* ) [}] ))
+        ''', re.VERBOSE)
+
+class ParserError(Parser.ParserError):
+    def __init__(self, error, tokens):
+        self.error = error # previous exception
+        self.tokens = tokens
+
+    def __str__(self):
+        lines = ['--- This code section failed: ---']
+        lines.extend( map(str, self.tokens) )
+        lines.extend( ['', str(self.error)] )
+        return '\n'.join(lines)
+
+
+__globals_tokens__ =  ('STORE_GLOBAL', 'DELETE_GLOBAL') # 'LOAD_GLOBAL'
+
+def find_globals(node, globals):
+    """Find globals in this statement."""
+    for n in node:
+        if isinstance(n, AST):
+            #if n != 'stmt': # skip nested statements
+                globals = find_globals(n, globals)
+        elif n.type in __globals_tokens__:
+            globals[n.pattr] = None
+    return globals
+
+
+class Walker(GenericASTTraversal, object):
+    stacked_params = ('f', 'indent', 'isLambda', '_globals')
+
+    def __init__(self, out, scanner, showast=0):
+        GenericASTTraversal.__init__(self, ast=None)
+        self.scanner = scanner
+        params = {
+            'f': out,
+            'indent': '',
+            }
+        self.showast = showast
+        self.__params = params
+        self.__param_stack = []
+
+    f = property(lambda s: s.__params['f'],
+                 lambda s, x: s.__params.__setitem__('f', x),
+                 lambda s: s.__params.__delitem__('f'),
+                 None)
+
+    indent = property(lambda s: s.__params['indent'],
+                 lambda s, x: s.__params.__setitem__('indent', x),
+                 lambda s: s.__params.__delitem__('indent'),
+                 None)
+
+    isLambda = property(lambda s: s.__params['isLambda'],
+                 lambda s, x: s.__params.__setitem__('isLambda', x),
+                 lambda s: s.__params.__delitem__('isLambda'),
+                 None)
+
+    _globals = property(lambda s: s.__params['_globals'],
+                 lambda s, x: s.__params.__setitem__('_globals', x),
+                 lambda s: s.__params.__delitem__('_globals'),
+                 None)
+
+    def indentMore(self, indent=TAB):
+        self.indent += indent
+    def indentLess(self, indent=TAB):
+        self.indent = self.indent[:-len(indent)]
+
+    def traverse(self, node, indent=None, isLambda=0):