Commits

Anonymous committed ec7dd95

Removing rope.base.oi.dynamicoi._ObjectPersistedForm

Comments (0)

Files changed (7)

docs/dev/issues.txt

 Hot Topics
 ==========
 
-* `Having Virtual PyModules`_
-* `Library Movement`_
+* `Having virtual PyModules`_
+* `Storing different return types for different set of arguments`_
 
 
 To Be Discussed
 
 * `Python's Implicit Interfaces`_
 * Removing `PythonRefactoring` facade
-* `Rope's preference system`_: format, location
+* `Rope's preference system`_
 * Indexing source files for faster occurrence finding
-* Having a persistant format for PyNames and PyObjects for saving
-  hard to compute information like class hierarchies
+* Saving hard to compute information like class hierarchies to files
 * Using a modification of `compiler` AST for simplifying refactorings
 * Undo Unification
 
 
+Storing Different Return Types For Different Set Of Arguments
+=============================================================
+
+Goals:
+
+* Better type inference for function return values and parameters
+* Better builtin type construction
+
+Problems:
+
+* Needs more memory
+* Slower module running
+* Slower object infers
+
+
+Sample Implementation
+---------------------
+
+Many places that use `PyFunction._get_returned_object` don't calculate
+argument types.  Change `rope.base.evalute` to do that.  What's more
+`PyFunction` caches ``returned_object`` which will no longer work if
+we perform this refactoring.  We can store its matching arguments to
+solve this problem.
+
+* Transforming `PyObject`\s to textual form for lookups
+* Change `PyFunction._get_returned_object` to get a parameter list
+* Change `dynamicoi` to store return value for each set of parameters
+* Change `ObjectInfer.infer_returned_object` and
+  `DynamicOI.infer_returned_object` to get parameters
+* `PyFunction`'s ``returned_object`` cache is no longer useful
+* Change calls to `_get_returned_object` in the `rope.base.evaluate`
+
+
+Needed Transformations:
+
+* Compiled -> Textual
+* Textual -> PyObject
+* PyObject -> Textual
+
+
+Considerations For A New Textual Form
+-------------------------------------
+
+Although textual forms are currently used only for transferring
+running data, it can be used for saving `PyObject`\s, too.  This
+might be useful if we want to save some of the collected information
+to a file.
+
+A textual form should:
+
+* be stored
+* have maximum immunity to changes
+* represent all kinds of objects
+
+The current textual form:
+
+* ('module', file)
+* ('function', file, lineno, (*args), returned)
+* ('class', file, name)
+* ('instance', file, name)
+* ('builtin', type, *args)
+* ('none')
+* ('unknown')
+
+Lot's of optimizations is possible.  But they make things more
+complicated.  For example By adding a file indexing form:
+
+* ('file', file, index)
+
+We can replace all files in above forms to use the integer indices
+specified with file command.  The main problem with this approach is
+that file indices might change across different runs.
+
+We can also remove 'none' and 'unknown' with and empty string.
+
+
 Rope's Preference System
 ========================
 
 
 * format
 
-  * human readable?
+  * human readable? yes
   * python (like emacs and lisp)
   
     - push or pull model for configurations
 * Inner functions
 
 
-Library Movement
-================
-
-* Adding `read` and `write` to `rope.base.fscommands`?
-
-
 Using ASTs For Transformations
 ==============================
 
 * Using static type inference algorithms
 
 
-Storing Different Return Types For Different Set Of Arguments
--------------------------------------------------------------
-
-Goals:
-
-* Better type inference for function return values and parameters
-* Better builtin type construction
-
-Problems:
-
-* Needs more memory
-* Slower object infers
-
-Implementation issues:
-
-* `runmod` does not need to be changed
-* Type of parameters and variables inside function
-
-
 Rejecting Smalltalk `RefactoringBrowser` Approach
 -------------------------------------------------
 
 a function is called, but we cannot always find the offset in that
 file from which this function is called.
 
+Although we cannot find the exact offset an occurrence happens we
+can know the suspected lines and that will help us shorten the
+scope for searching for occurrences considerably.
+
 If we solve these problems we can can use a strategy for finding
 occurrences.  Also Think of other ways of collecting type information.
 
 -----------------
 
 Rope should have a good emacs-ish keybinding.
+
+* New ...; C-x n ...
+
+  * C-x n m: new module
+  * C-x n p: new package
+  * C-x n f: new file
+  * C-x n d: new directory
+
+* Version control; C-x v ...
+* Refactorings; C-c r ...
+
+  * C-x r r: rename
+  * C-x r m: extract method
+  * C-x r i: inline
+  * C-x r l: extract local
+  * C-x r c: change signature
+
+* Help; C-h ...

docs/dev/workingon.txt

 Small Stories
 =============
 
-- adding builtin `str` and `unicode`
-- handling extract method indented module region
-- difference between builtin types and their instances
+- removing `dynamicoi._ObjectPersistedForm`
 
 * docs for builtin functions
 * exploring module running
 
-  * check the reason for speed difference; is it caused by fifo and
-    socket issues?
   * check the amount useful information transfered
   * possible techniques: compression
 

rope/base/builtins.py

         self.holding = holding
         self.attributes = {
             '__getitem__': BuiltinName(BuiltinFunction(self.holding)),
-            '__getslice__': BuiltinName(BuiltinFunction(self)),
+            '__getslice__': BuiltinName(BuiltinFunction(pyobjects.PyObject(self))),
             'pop': BuiltinName(BuiltinFunction(self.holding)),
             '__iter__': BuiltinName(BuiltinFunction(Iterator(self.holding))),
             'append': BuiltinName(BuiltinFunction()),
             'itervalues': BuiltinName(Iterator(self.values)),
             'items': BuiltinName(List(item)),
             'iteritems': BuiltinName(Iterator(item)),
-            'copy': BuiltinName(BuiltinFunction(self)),
+            'copy': BuiltinName(BuiltinFunction(pyobjects.PyObject(self))),
             'clear': BuiltinName(BuiltinFunction()),
             'has_key': BuiltinName(BuiltinFunction()),
             'popitem': BuiltinName(BuiltinFunction()),
             first = objects[0]
         self.attributes = {
             '__getitem__': BuiltinName(BuiltinFunction(first)),
-            '__getslice__': BuiltinName(BuiltinFunction(self)),
+            '__getslice__': BuiltinName(BuiltinFunction(pyobjects.PyObject(self))),
             '__iter__': BuiltinName(BuiltinFunction(Iterator(first)))}
     
     def get_holding_objects(self):
             'pop': BuiltinName(BuiltinFunction(self.holding)),
             '__iter__': BuiltinName(BuiltinFunction(Iterator(self.holding))),
             'add': BuiltinName(BuiltinFunction()),
-            'copy': BuiltinName(BuiltinFunction(self)),
-            'difference': BuiltinName(BuiltinFunction(self)),
-            'intersection': BuiltinName(BuiltinFunction(self)),
+            'copy': BuiltinName(BuiltinFunction(pyobjects.PyObject(self))),
+            'difference': BuiltinName(BuiltinFunction(pyobjects.PyObject(self))),
+            'intersection': BuiltinName(BuiltinFunction(pyobjects.PyObject(self))),
             'difference_update': BuiltinName(BuiltinFunction()),
-            'symmetric_difference': BuiltinName(BuiltinFunction(self)),
+            'symmetric_difference': BuiltinName(BuiltinFunction(pyobjects.PyObject(self))),
             'symmetric_difference_update': BuiltinName(BuiltinFunction()),
-            'union': BuiltinName(BuiltinFunction(self)),
+            'union': BuiltinName(BuiltinFunction(pyobjects.PyObject(self))),
             'discard': BuiltinName(BuiltinFunction()),
             'remove': BuiltinName(BuiltinFunction()),
             'issuperset': BuiltinName(BuiltinFunction()),
     
     def __init__(self):
         super(Str, self).__init__(pyobjects.PyObject.get_base_type('Type'))
+        self_object = pyobjects.PyObject(self)
         self.attributes = {
-            '__getitem__': BuiltinName(BuiltinFunction(self)),
-            '__getslice__': BuiltinName(BuiltinFunction(self)),
-            '__iter__': BuiltinName(BuiltinFunction(Iterator(self))),
-            'captialize': BuiltinName(BuiltinFunction(self)),
-            'center': BuiltinName(BuiltinFunction(self)),
+            '__getitem__': BuiltinName(BuiltinFunction(self_object)),
+            '__getslice__': BuiltinName(BuiltinFunction(self_object)),
+            '__iter__': BuiltinName(BuiltinFunction(Iterator(self_object))),
+            'captialize': BuiltinName(BuiltinFunction(self_object)),
+            'center': BuiltinName(BuiltinFunction(self_object)),
             'count': BuiltinName(BuiltinFunction()),
-            'decode': BuiltinName(BuiltinFunction(self)),
-            'encode': BuiltinName(BuiltinFunction(self)),
+            'decode': BuiltinName(BuiltinFunction(self_object)),
+            'encode': BuiltinName(BuiltinFunction(self_object)),
             'endswith': BuiltinName(BuiltinFunction()),
-            'expandtabs': BuiltinName(BuiltinFunction(self)),
+            'expandtabs': BuiltinName(BuiltinFunction(self_object)),
             'find': BuiltinName(BuiltinFunction()),
             'index': BuiltinName(BuiltinFunction()),
             'isalnum': BuiltinName(BuiltinFunction()),
             'isspace': BuiltinName(BuiltinFunction()),
             'istitle': BuiltinName(BuiltinFunction()),
             'isupper': BuiltinName(BuiltinFunction()),
-            'join': BuiltinName(BuiltinFunction(self)),
-            'ljust': BuiltinName(BuiltinFunction(self)),
-            'lower': BuiltinName(BuiltinFunction(self)),
-            'lstrip': BuiltinName(BuiltinFunction(self)),
-            'replace': BuiltinName(BuiltinFunction(self)),
+            'join': BuiltinName(BuiltinFunction(self_object)),
+            'ljust': BuiltinName(BuiltinFunction(self_object)),
+            'lower': BuiltinName(BuiltinFunction(self_object)),
+            'lstrip': BuiltinName(BuiltinFunction(self_object)),
+            'replace': BuiltinName(BuiltinFunction(self_object)),
             'rfind': BuiltinName(BuiltinFunction()),
             'rindex': BuiltinName(BuiltinFunction()),
-            'rjust': BuiltinName(BuiltinFunction(self)),
-            'rsplit': BuiltinName(BuiltinFunction(self)),
-            'rstrip': BuiltinName(BuiltinFunction(self)),
-            'split': BuiltinName(BuiltinFunction(self)),
-            'splitlines': BuiltinName(BuiltinFunction(self)),
-            'startswith': BuiltinName(BuiltinFunction(self)),
-            'strip': BuiltinName(BuiltinFunction(self)),
-            'swapcase': BuiltinName(BuiltinFunction(self)),
-            'title': BuiltinName(BuiltinFunction(self)),
-            'translate': BuiltinName(BuiltinFunction(self)),
-            'upper': BuiltinName(BuiltinFunction(self)),
-            'zfill': BuiltinName(BuiltinFunction(self))}
+            'rjust': BuiltinName(BuiltinFunction(self_object)),
+            'rsplit': BuiltinName(BuiltinFunction(get_list(self_object))),
+            'rstrip': BuiltinName(BuiltinFunction(self_object)),
+            'split': BuiltinName(BuiltinFunction(get_list(self_object))),
+            'splitlines': BuiltinName(BuiltinFunction(get_list(self_object))),
+            'startswith': BuiltinName(BuiltinFunction(self_object)),
+            'strip': BuiltinName(BuiltinFunction(self_object)),
+            'swapcase': BuiltinName(BuiltinFunction(self_object)),
+            'title': BuiltinName(BuiltinFunction(self_object)),
+            'translate': BuiltinName(BuiltinFunction(self_object)),
+            'upper': BuiltinName(BuiltinFunction(self_object)),
+            'zfill': BuiltinName(BuiltinFunction(self_object))}
     
     def get_attributes(self):
         return self.attributes

rope/base/oi/dynamicoi.py

     def __init__(self, pycore):
         self.pycore = pycore
         self.files = {}
+        self.to_pyobject = _TextualToPyObject(self.pycore.project)
     
     def run_module(self, resource, args=None, stdin=None, stdout=None):
         """Return a PythonFileRunner for controlling the process"""
     def infer_returned_object(self, pyobject):
         organizer = self._find_organizer(pyobject)
         if organizer:
-            return organizer.returned.to_pyobject(self.pycore.project)
+            return self.to_pyobject.transform(organizer.returned)
 
     def infer_parameter_objects(self, pyobject):
         organizer = self._find_organizer(pyobject)
         if organizer and organizer.args is not None:
-            pyobjects = [parameter.to_pyobject(self.pycore.project)
+            pyobjects = [self.to_pyobject.transform(parameter)
                          for parameter in organizer.args]
             return pyobjects
     
             self.files[path] = {}
         if lineno not in self.files[path]:
             self.files[path][lineno] = _CallInformationOrganizer()
-        returned = _ObjectPersistedForm.create_persistent_object(data[2])
-        args = [_ObjectPersistedForm.create_persistent_object(arg) for arg in data[1]]
-        self.files[path][lineno].add_call_information(args, returned)
+        self.files[path][lineno].add_call_information(data[1], data[2])
 
 
 class _CallInformationOrganizer(object):
         self.returned = None
     
     def add_call_information(self, args, returned):
-        if self.returned is None or \
-           not isinstance(returned, (_PersistedNone, _PersistedUnknown)):
+        if self.returned is None or (args and args[0][0] not in ('unknown', 'none')):
+            self.args = args
+        if self.returned is None or returned[0] not in ('unknown', 'none'):
             self.returned = returned
-        if self.returned is None or args and \
-           not isinstance(args[0], (_PersistedNone, _PersistedUnknown)):
-            self.args = args
 
 
-class _ObjectPersistedForm(object):
+class _TextualToPyObject(object):
     
-    def _get_pymodule(self, project, path):
-        root = os.path.abspath(project.get_root_address())
+    def __init__(self, project):
+        self.project = project
+    
+    def transform(self, textual):
+        """Transform an object from textual form to `PyObject`"""
+        type = textual[0]
+        method = getattr(self, type + '_to_pyobject')
+        return method(textual)
+
+    def module_to_pyobject(self, textual):
+        path = textual[1]
+        return self._get_pymodule(path)
+    
+    def builtin_to_pyobject(self, textual):
+        name = textual[1]
+        if name == 'str':
+            return builtins.get_str()
+        if name == 'list':
+            holding = self.transform(textual[2])
+            return builtins.get_list(holding)
+        if name == 'dict':
+            keys = self.transform(textual[2])
+            values = self.transform(textual[3])
+            return builtins.get_dict(keys, values)
+        if name == 'tuple':
+            objects = []
+            for holding in textual[2:]:
+                objects.append(self.transform(holding))
+            return builtins.get_tuple(*objects)
+        if name == 'set':
+            holding = self.transform(textual[2])
+            return builtins.get_set(holding)
+        return None
+    
+    def unknown_to_pyobject(self, textual):
+        return None
+    
+    def none_to_pyobject(self, textual):
+        return None
+    
+    def function_to_pyobject(self, textual):
+        return self._get_pyobject_at(textual[1], textual[2])
+    
+    def class_to_pyobject(self, textual):
+        path, name = textual[1:]
+        pymodule = self._get_pymodule(path)
+        module_scope = pymodule.get_scope()
+        suspected_pyobject = None
+        if name in module_scope.get_names():
+            suspected_pyobject = module_scope.get_name(name).get_object()
+        if suspected_pyobject is not None and \
+           suspected_pyobject.get_type() == pyobjects.PyObject.get_base_type('Type'):
+            return suspected_pyobject
+        else:
+            lineno = self._find_occurrence(name, pymodule.get_resource().read())
+            if lineno is not None:
+                inner_scope = module_scope.get_inner_scope_for_line(lineno)
+                return inner_scope.pyobject
+    
+    def instance_to_pyobject(self, textual):
+        return pyobjects.PyObject(self.class_to_pyobject(textual))
+    
+    def _find_occurrence(self, name, source):
+        pattern = re.compile(r'^\s*class\s*' + name + r'\b')
+        lines = source.split('\n')
+        for i in range(len(lines)):
+            if pattern.match(lines[i]):
+                return i + 1
+
+    def _get_pymodule(self, path):
+        root = os.path.abspath(self.project.get_root_address())
         if path.startswith(root):
             relative_path = path[len(root):]
             if relative_path.startswith('/') or relative_path.startswith(os.sep):
                 relative_path = relative_path[1:]
-            resource = project.get_resource(relative_path)
+            resource = self.project.get_resource(relative_path)
         else:
-            resource = project.get_out_of_project_resource(path)
-        return project.get_pycore().resource_to_pyobject(resource)
+            resource = self.project.get_out_of_project_resource(path)
+        return self.project.get_pycore().resource_to_pyobject(resource)
     
-    def _get_pyobject_at(self, project, path, lineno):
-        scope = self._get_pymodule(project, path).get_scope()
+    def _get_pyobject_at(self, path, lineno):
+        scope = self._get_pymodule(path).get_scope()
         inner_scope = scope.get_inner_scope_for_line(lineno)
         return inner_scope.pyobject
 
-    # TODO: Implement __eq__ for subclasses
-    def __eq__(self, object_):
-        if type(object) != type(self):
-            return False
-
-    @staticmethod
-    def create_persistent_object(data):
-        type_ = data[0]
-        if type_ == 'none':
-            return _PersistedNone()
-        if type_ == 'module':
-            return _PersistedModule(*data[1:])
-        if type_ == 'function':
-            return _PersistedFunction(*data[1:])
-        if type_ == 'class':
-            return _PersistedClass(*data[1:])
-        if type_ == 'builtin':
-            return _PersistedBuiltin(*data[1:])
-        if type_ == 'instance':
-            return _PersistedClass(is_instance=True, *data[1:])
-        return _PersistedUnknown()
-
-
-class _PersistedNone(_ObjectPersistedForm):
-
-    def to_pyobject(self, project):
-        return None
-
-
-class _PersistedUnknown(_ObjectPersistedForm):
-
-    def to_pyobject(self, project):
-        return None
-
-
-class _PersistedBuiltin(_ObjectPersistedForm):
-    
-    def __init__(self, name, *data):
-        self.name = name
-        self.data = data
-
-    def to_pyobject(self, project):
-        if self.name == 'str':
-            return builtins.get_str()
-        if self.name == 'list':
-            holding = _ObjectPersistedForm.create_persistent_object(self.data[0])
-            return builtins.get_list(holding.to_pyobject(project))
-        if self.name == 'dict':
-            keys = _ObjectPersistedForm.create_persistent_object(self.data[0])
-            values = _ObjectPersistedForm.create_persistent_object(self.data[1])
-            return builtins.get_dict(keys.to_pyobject(project),
-                                     values.to_pyobject(project))
-        if self.name == 'tuple':
-            objects = []
-            for holding in self.data:
-                objects.append(_ObjectPersistedForm.
-                               create_persistent_object(holding).to_pyobject(project))
-            return builtins.get_tuple(*objects)
-        if self.name == 'set':
-            holding = _ObjectPersistedForm.create_persistent_object(self.data[0])
-            return builtins.get_set(holding)
-        return None
-
-
-class _PersistedModule(_ObjectPersistedForm):
-    
-    def __init__(self, path):
-        self.path = path
-    
-    def to_pyobject(self, project):
-        return self._get_pymodule(project, self.path)
-
-
-class _PersistedFunction(_ObjectPersistedForm):
-    
-    def __init__(self, path, lineno):
-        self.path = path
-        self.lineno = lineno
-    
-    def to_pyobject(self, project):
-        return self._get_pyobject_at(project, self.path, self.lineno)
-
-
-class _PersistedClass(_ObjectPersistedForm):
-    
-    def __init__(self, path, name, is_instance=False):
-        self.path = path
-        self.name = name
-        self.is_instance = is_instance
-    
-    def to_pyobject(self, project):
-        pymodule = self._get_pymodule(project, self.path)
-        module_scope = pymodule.get_scope()
-        suspected_pyobject = None
-        if self.name in module_scope.get_names():
-            suspected_pyobject = module_scope.get_name(self.name).get_object()
-        if suspected_pyobject is not None and \
-           suspected_pyobject.get_type() == pyobjects.PyObject.get_base_type('Type'):
-            if self.is_instance:
-                return pyobjects.PyObject(suspected_pyobject)
-            else:
-                return suspected_pyobject
-        else:
-            lineno = self._find_occurrence(pymodule.get_resource().read())
-            if lineno is not None:
-                inner_scope = module_scope.get_inner_scope_for_line(lineno)
-                return inner_scope.pyobject
-    
-    def _find_occurrence(self, source):
-        pattern = re.compile(r'^\s*class\s*' + self.name + r'\b')
-        lines = source.split('\n')
-        for i in range(len(lines)):
-            if pattern.match(lines[i]):
-                return i + 1
-
 
 class PythonFileRunner(object):
     """A class for running python project files"""

rope/base/oi/objectinfer.py

 
     def infer_returned_object(self, pyobject):
         """Infers the `PyObject` this callable `PyObject` returns after calling"""
-        dynamically_inferred_object = self.pycore.dynamicoi.infer_returned_object(pyobject)
-        if dynamically_inferred_object is not None:
-            return dynamically_inferred_object
+        dynamically_inferred = self.pycore.dynamicoi.\
+                               infer_returned_object(pyobject)
+        if dynamically_inferred is not None:
+            return dynamically_inferred
         scope = pyobject.get_scope()
         if not scope._get_returned_asts():
             return

rope/base/oi/runmod.py

             if event != 'return':
                 return
             args = []
-            returned = ('unknown')
+            returned = ('unknown',)
             code = frame.f_code
             for argname in code.co_varnames[:code.co_argcount]:
                 try:
                     args.append(self._object_to_persisted_form(frame.f_locals[argname]))
                 except (TypeError, AttributeError):
-                    args.append(('unknown'))
+                    args.append(('unknown',))
             try:
                 returned = self._object_to_persisted_form(arg)
             except (TypeError, AttributeError):
                 return (type_, os.path.abspath(inspect.getsourcefile(object_)),
                         object_.__name__)
             except (TypeError, AttributeError):
-                return ('unknown')
+                return ('unknown',)
     
         def _get_persisted_builtin(self, object_):
             if isinstance(object_, (str, unicode)):
                         holding = o
                         break
                 return ('builtin', 'set', self._object_to_persisted_form(holding))
-            return ('unknown')
+            return ('unknown',)
     
         def _object_to_persisted_form(self, object_):
             if object_ == None:
-                return ('none')
+                return ('none',)
             if isinstance(object_, types.CodeType):
                 return self._get_persisted_code(object_)
             if isinstance(object_, types.FunctionType):

rope/ui/codeassist.py

 actions = []
 
 actions.append(SimpleAction('Code Assist', do_code_assist, 'M-slash',
-                            MenuAddress(['Code', 'Code Assist'], 'c'), ['python']))
+                            MenuAddress(['Code', 'Code Assist (Auto-Complete)'], 'c'), ['python']))
 actions.append(SimpleAction('Goto Definition', do_goto_definition, 'F3',
                             MenuAddress(['Code', 'Goto Definition'], 'g'), ['python']))
 actions.append(SimpleAction('Show Doc', do_show_doc, 'F2',