Commits

Ali Gholami Rudi  committed 7bddb54

Renaming occurrences in strings and comments

  • Participants
  • Parent commits 66ba7f2

Comments (0)

Files changed (16)

 New Features
 ============
 
-* Stoppable refactorings
-* Basic implicit interfaces
-* Spell-Checker
-* Automatic SOI analysis
+* `Stoppable refactorings`_
+* `Basic implicit interfaces`_
+* `Spell-Checker`_
+* `Automatic SOI analysis`_
+* `Renaming occurrences in strings and comments`_
+* Faster occurrence finding
+
+Basic Implicit Interfaces
+-------------------------
 
 Implicit interfaces are the interfaces that you don't explicitly
 define; But you expect a group of classes to have some common
   count_for(B())
 
 
-This also works for change method signature, too.  Note that this
-feature relies on rope's object inference mechanisms to find out the
-parameters that are passed to a function.  Also see the automatic
-SOI analysis added in this release.
+This also works for change method signature.  Note that this feature
+relies on rope's object inference mechanisms to find out the
+parameters that are passed to a function.  Also see the `automatic SOI
+analysis`_ that is added in this release.
 
-Note that rope already supports changing attributes across class
-hierarchies(explicit interfaces) and in this release basic implicit
-interfaces are handled for rename and change method signature
-refactorings.
+Stoppable Refactorings
+----------------------
 
 Another notable new feature is stoppable refactorings.  Some
 refactorings might take a long time to finish (based on the size of
 taskhandle.TaskHandle` to this method.  See `rope.refactor.taskhandle`
 module for more information.
 
+Automatic SOI Analysis
+----------------------
+
 Maybe the most important internal feature added in this release is
 automatic static object inference analysis.  When turned on, it
 analyzes the changed scopes of a file when saving for obtaining object
 ``${your_project_root}/.ropeproject/config.py``, if you're new to
 rope), though that is not recommended.
 
+Renaming Occurrences In Strings And Comments
+--------------------------------------------
+
+You can tell rope to rename all occurrences of a name in comments and
+strings.  This can be done in the rename dialog by selecting its radio
+button or by passing ``docs=True`` to `Rename.get_changes()` method
+when using rope as a library.  Rope renames names in comments and
+strings only when the name is visible there.  For example in::
+
+  def f():
+      a_var = 1
+      print 'a_var = %s' % a_var
+
+  # f prints a_var
+
+after we rename the `a_var` local variable in `f()` to `new_var` we
+would get::
+
+  def f():
+      new_var = 1
+      print 'new_var = %s' % new_var
+
+  # f prints a_var
+
+This makes it safe to assume that this option does not perform wrong
+renames most of the time and for this reason it is by default on in
+the UI (though not in `Rename.get_changes()`).
+
+Spell-Checker
+-------------
+
 The new spell-checker uses ispell/aspell if available.  You can use
 ``M-$`` like emacs for checking current word.  You can also use ``C-x
 $ r`` and ``C-x $ b`` for spell-checking region and buffer.

File docs/dev/done.txt

 ===========
 
 
+- Encapsulating field in the defining class : April 13, 2007
+
+
+- Renaming occurrences in strings and comments : April 13, 2007
+
+
 - Stoppable refactorings : April 11, 2007
 
 

File docs/dev/issues.txt

 decisions.
 
 
+Introducing `TemporaryScope`
+============================
+
+There are some language structures that have their own separate scope
+like lambdas, list comprehensions and generator expressions.
+
+These scopes does not have any nested scopes and their parent scopes
+do not include them as their sub-scopes.  The main character of these
+scopes is that the names defined there are not visible from outside.
+What's more they are a single expression.
+
+Lambda have a longer story.  We want the SOI to be able to infer their
+returned value.  We don't want DOI for lambdas.
+
+
+Finding Class Hierarchies
+=========================
+
+Currently for finding the classes in a project all source files are
+searched and compiled.  This takes a lot of time.  What's more when a
+module is invalidated class cache is cleaned up.  This means that
+we have to recaclulate classes each time.
+
+* We can save classes to a file so that we can use them across
+  sessions.  But that is not a good solution because detecting changes
+  to modules and handling these information on disk needs lots of
+  work.
+* Now that creating this cache is so expensive we can at least
+  calculate the hierachies only once for each session and updating it
+  when files change.
+* Since almost all modules contain at least one class, for creating
+  all classes we need to compile all modules.
+* We can visit the AST of a module instead of textually searching the
+  source code; this might improve the speed.
+
+::
+
+  class _ClassCacher(object):
+      """Caches classes inside the project"""
+
+
 Better SOI
 ==========
 
 * Validating call infos
 
 
-Validating Information
-----------------------
-
-When do we need to recalculate the information? Currently rope does
-not save anything unless an exact object is calculated; in that case
-it will be used until they remain valid.  The problems with this
-approach includes:
-
-* The method might be changed in future.
-
-  * Returned object
-  * Parameter objects
-  * The number of parameters
-
-* We don't want to recalculate 'unknown's and 'none's every time
-* The methods or data accessed by the method might change
-
-The need for invalidating data is felt more after saving object data
-to disk.
-
-
 Saving Data On Disk
 -------------------
 

File docs/dev/stories.txt

 ==================
 
 * Editor folding
-* Variable indentation and tab size
 * Auto-importing modules
 * Replacement; M-%
 * Remembering last open project
 * Handling the return type of ``yield`` keyword
 
 
-* Encapsulate field using properties
-
-
 * Extracting methods from pieces with only one return/yield statement
 
 
-* Asking whether to encapsulate field in the class itself
-
-
 * Moving initialization to constructor in local variable to field
 
 
+* Encapsulate field using properties
+
+
 * Performing import actions only on one import statement
 
 
 * Inlining a single occurrence
 
 
-* Renaming textual matches
-
-
 * Searching inside `EnhancedList`\s
 
 
 * Enhancing open project dialog
 
 
+* Inferring the object list comprehensions or generator expressions hold
+
+
 > Public Release 0.5m5 : April 15, 2007
-
-
-* Inferring the object list comprehensions or generator expressions hold

File docs/dev/workingon.txt

 Small Stories
 =============
 
+- Faster occurrence finding; long time for re matching
+- Renaming occurrences in strings and comments
+- Encapsulating field in the defining class
+- Problem finding start of block when there are generator expressions
+  containing for or if at the start of line
+
+* Long rename in hierarchy; it compiles all files; pycore needs refactorings
 * Validating callinfo
 * Codeassist only shows saved data
 * ObjectDB after change method signature

File docs/index.txt

   * Convert local variable to field
   * Replace method with method object
   * Inline argument default value
-  * Handling a basic form of implicit interfaces
+  * Basic implicit interfaces handling
   * Previewing refactorings
   * Stopping refactorings
   * Undo/redo refactorings

File rope/base/codeanalyze.py

         if match is not None and \
            count_line_indents(lines.get_line(i)) <= maximum_indents:
             striped = match.string.lstrip()
-            # Maybe we're in a list comprehension
+            # Maybe we're in a list comprehension or generator expression
             if i > 1 and striped.startswith('if') or striped.startswith('for'):
                 bracs = 0
                 for j in range(i, min(i + 5, lines.length() + 1)):
                     for c in lines.get_line(j):
                         if c == '#':
                             break
-                        if c == '[':
+                        if c in '[(':
                             bracs += 1
-                        if c == ']':
+                        if c in ')]':
                             bracs -= 1
                             if bracs < 0:
                                 break

File rope/base/pycore.py

         pymodule = self.resource_to_pyobject(resource)
         pymodule._invalidate_concluded_data()
         self.object_infer.soi.analyze_module(pymodule, should_analyze)
-        pymodule._invalidate_concluded_data()
 
     def get_subclasses(self, pyclass):
         if self.classes is None:
             classes = []
             pattern = re.compile(r'^[ \t]*class[ \t]+\w', re.M)
             for resource in self.get_python_files():
-                pyscope = self.resource_to_pyobject(resource).get_scope()
+                try:
+                    pyscope = self.resource_to_pyobject(resource).get_scope()
+                except SyntaxError:
+                    continue
                 source = pyscope.pyobject.source_code
                 for match in pattern.finditer(source):
                     holding_scope = pyscope.get_inner_scope_for_offset(match.start())

File rope/ide/spellchecker.py

 
     def check(self):
         lines = self.text.splitlines()
-        try:
-            for line in lines:
-                if self.do_quit:
-                    break
-                for typo in self._check_line(line):
-                    yield typo
-                self.line_offset += len(line) + 1
-                self.line_ignored.clear()
-        finally:
-            if self.save_dict:
-                self.aspell.write_line('#')
-            self.aspell.close()
+        for line in lines:
+            if self.do_quit:
+                break
+            for typo in self._check_line(line):
+                yield typo
+            self.line_offset += len(line) + 1
+            self.line_ignored.clear()
+        # PORT: Removed finally clause for python 2.4
+        if self.save_dict:
+            self.aspell.write_line('#')
+        self.aspell.close()
 
     def _check_line(self, line):
         self.aspell.write_line('^%s' % line)

File rope/refactor/encapsulate_field.py

         return changes
 
     def _get_defining_class_scope(self):
-        defining_pymodule, defining_line = self.pyname.get_definition_location()
-        defining_scope = defining_pymodule.get_scope().get_inner_scope_for_line(defining_line)
+        defining_scope = self._get_defining_scope()
         if defining_scope.get_kind() == 'Function':
             defining_scope = defining_scope.parent
         return defining_scope
 
+    def _get_defining_scope(self):
+        pymodule, line = self.pyname.get_definition_location()
+        return pymodule.get_scope().get_inner_scope_for_line(line)
+
     def _change_holding_module(self, changes, rename_in_module):
         pymodule = self.pycore.resource_to_pyobject(self.resource)
         class_scope = self._get_defining_class_scope()
-        class_start_line = class_scope.get_start()
-        class_end_line = class_scope.get_end()
-        class_start = pymodule.lines.get_line_start(class_start_line)
-        class_end = pymodule.lines.get_line_end(class_end_line)
-        new_source = rename_in_module.get_changed_module(pymodule=pymodule,
-                                                         skip_start=class_start,
-                                                         skip_end=class_end)
+        defining_object = self._get_defining_scope().pyobject
+        skip_start, skip_end = sourceutils.get_body_region(defining_object)
+        new_source = rename_in_module.get_changed_module(
+            pymodule=pymodule, skip_start=skip_start, skip_end=skip_end)
         if new_source is not None:
             pymodule = self.pycore.get_string_module(new_source, self.resource)
-            class_scope = pymodule.get_scope().get_inner_scope_for_line(class_start_line)
-        getter = 'def get_%s(self):\n    return self.%s' % (self.name, self.name)
-        setter = 'def set_%s(self, value):\n    self.%s = value' % (self.name, self.name)
+            class_scope = pymodule.get_scope().\
+                          get_inner_scope_for_line(class_scope.get_start())
+        indents = sourceutils.get_indent(self.pycore) * ' '
+        getter = 'def get_%s(self):\n%sreturn self.%s' % \
+                 (self.name, indents, self.name)
+        setter = 'def set_%s(self, value):\n%sself.%s = value' % \
+                 (self.name, indents, self.name)
         new_source = sourceutils.add_methods(pymodule, class_scope,
                                              [getter, setter])
         changes.add_change(ChangeContents(pymodule.get_resource(), new_source))

File rope/refactor/occurrences.py

 class OccurrenceFinder(object):
     """For finding textual occurrences of a name"""
 
-    def __init__(self, pycore, name):
+    def __init__(self, pycore, name, docs=False):
         self.pycore = pycore
         self.name = name
+        self.docs = docs
         self.comment_pattern = OccurrenceFinder.any('comment', [r'#[^\n]*'])
         sqstring = r"(\b[rR])?'[^'\\\n]*(\\.[^'\\\n]*)*'?"
         dqstring = r'(\b[rR])?"[^"\\\n]*(\\.[^"\\\n]*)*"?'
     def find_occurrences(self, resource=None, pymodule=None):
         """Generate `Occurrence` instances"""
         tools = _OccurrenceToolsCreator(self.pycore, resource, pymodule)
-        for match in self.pattern.finditer(tools.source_code):
+        if not self._fast_file_query(tools.source_code):
+            return
+        if self.docs:
+            searcher = self._normal_search
+        else:
+            searcher = self._re_search
+        for matched in searcher(tools.source_code):
+            yield Occurrence(tools, matched + 1)
+
+    def _re_search(self, source):
+        for match in self.pattern.finditer(source):
             for key, value in match.groupdict().items():
                 if value and key == 'occurrence':
-                    yield Occurrence(tools, match.start(key) + 1)
+                    yield match.start(key)
+
+    def _normal_search(self, source):
+        current = 0
+        while True:
+            try:
+                found = source.index(self.name, current)
+                current = found + len(self.name)
+                if (found == 0 or not self._is_id_char(source[found - 1])) and \
+                   (current == len(source) or not self._is_id_char(source[current])):
+                    yield found
+            except ValueError:
+                break
+
+    def _is_id_char(self, c):
+        return c.isalnum() or c == '_'
+
+    def _fast_file_query(self, source):
+        try:
+            source.index(self.name)
+            return True
+        except ValueError:
+            return False
 
     def _get_source(self, resource, pymodule):
         if resource is not None:
 class FilteredFinder(object):
 
     def __init__(self, pycore, name, pynames, only_calls=False,
-                 imports=True, unsure=False):
+                 imports=True, unsure=False, docs=False):
         self.pycore = pycore
         self.pynames = pynames
         self.name = name
         self.only_calls = only_calls
         self.imports = imports
         self.unsure = unsure
-        self.occurrence_finder = OccurrenceFinder(pycore, name)
+        self.occurrence_finder = OccurrenceFinder(pycore, name, docs=docs)
 
     def find_occurrences(self, resource=None, pymodule=None):
         for occurrence in self.occurrence_finder.find_occurrences(

File rope/refactor/rename.py

         return self.old_name
 
     def get_changes(self, new_name, in_file=False, in_hierarchy=False,
-                    unsure=False, task_handle=taskhandle.NullTaskHandle()):
+                    unsure=False, docs=False,
+                    task_handle=taskhandle.NullTaskHandle()):
         """Get the changes needed for this refactoring
 
         :parameters:
               passed resource.
             - `in_hierarchy`: when renaming a method this keyword forces
               to rename all matching methods in the hierarchy
+            - `docs`: when `True` rename refactoring will rename
+              occurrences in comments and strings where the name is
+              visible.
             - `unsure`: rename occurrence even if unsure.  Do not use it
               unless you know what you're doing.  If `True`, all name
               occurrences except those that we are sure are not what we
         files = self._get_interesting_files(in_file)
         changes = ChangeSet('Renaming <%s> to <%s>' %
                             (self.old_name, new_name))
-        finder = occurrences.FilteredFinder(self.pycore, self.old_name,
-                                            old_pynames, unsure=unsure)
+        finder = occurrences.FilteredFinder(
+            self.pycore, self.old_name, old_pynames, unsure=unsure, docs=docs)
         job_set = task_handle.create_job_set('Collecting Changes', len(files))
         for file_ in files:
             job_set.started_job('Working on <%s>' % file_.path)

File rope/ui/refactor.py

         new_name = self.new_name_entry.get()
         return self.renamer.get_changes(
             new_name, in_file=self.is_local,
-            in_hierarchy=self.in_hierarchy.get(),
-            unsure=self.unsure.get(), task_handle=handle)
+            in_hierarchy=self.in_hierarchy.get(), unsure=self.unsure.get(),
+            docs=self.docs.get(), task_handle=handle)
 
     def _get_dialog_frame(self):
         frame = Tkinter.Frame(self.toplevel)
         self.new_name_entry.bind('<Return>', lambda event: self._ok())
         self.in_hierarchy = Tkinter.IntVar()
         self.unsure = Tkinter.IntVar()
+        self.docs = Tkinter.IntVar()
+        self.docs.set(1)
         in_hierarchy = Tkinter.Checkbutton(
             frame, text='Do for all matching methods in class hierarchy',
             variable=self.in_hierarchy)
         unsure = Tkinter.Checkbutton(
             frame, text='Rename when unsure(Know what you\'re doing!)',
             variable=self.unsure)
+        docs = Tkinter.Checkbutton(
+            frame, text='Rename occurrences in strings and comments where the name is visible',
+            variable=self.docs)
         index = 1
         if self.renamer.is_method():
             in_hierarchy.grid(row=1, columnspan=2, sticky=Tkinter.W)
             index += 1
         unsure.grid(row=index, columnspan=2, sticky=Tkinter.W)
+        index += 1
+        docs.grid(row=index, columnspan=2, sticky=Tkinter.W)
         self.new_name_entry.focus_set()
         return frame
 

File ropetest/codeanalyzetest.py

         line_finder = self._get_logical_line_finder(code)
         self.assertEquals((1, 2), line_finder.get_logical_line_in(2))
 
+    def test_generator_expressions_and_fors(self):
+        code = 'a_list = (i\n    for i in range(10))\n'
+        line_finder = self._get_logical_line_finder(code)
+        self.assertEquals((1, 2), line_finder.get_logical_line_in(2))
+
     def test_fors_and_block_start(self):
         code = 'l = range(10)\nfor i in l:\n    print i\n'
         self.assertEquals(2, get_block_start(SourceLinesAdapter(code),2 ))

File ropetest/refactor/__init__.py

         self.a_class = 'class A(object):\n    def __init__(self):\n        self.attr = 1\n'
         self.setter_and_getter = '\n    def get_attr(self):\n        return self.attr\n\n' \
                                  '    def set_attr(self, value):\n        self.attr = value\n'
-        self.encapsulated = 'class A(object):\n    def __init__(self):\n        self.attr = 1\n\n' \
-                            '    def get_attr(self):\n        return self.attr\n\n' \
-                            '    def set_attr(self, value):\n        self.attr = value\n'
+        self.encapsulated = self.a_class + self.setter_and_getter
 
     def tearDown(self):
         testutils.remove_project(self.project)
             'import mod\na_var = mod.A()\na_var.set_attr(a_var.get_attr() << 1)\n',
             self.mod1.read())
 
+    def test_changing_occurrences_inside_the_class(self):
+        new_class = self.a_class + '\n    def a_func(self):\n        self.attr = 1\n'
+        self.mod.write(new_class)
+        self._perform_encapsulate_field(self.mod, self.mod.read().index('attr') + 1)
+        expected = self.a_class + '\n    def a_func(self):\n        self.set_attr(1)\n' + \
+                   self.setter_and_getter
+        self.assertEquals(expected, self.mod.read())
+
 
 class LocalToFieldTest(unittest.TestCase):
 

File ropetest/refactor/renametest.py

             'c1 = C1()\nc1.new_func()\nc2 = C2()\nc2.a_func()\n',
             mod1.read())
 
+    def test_renaming_in_strings_and_comments(self):
+        code = 'a_var = 1\n# a_var\n'
+        mod1 = self.pycore.create_module(self.project.root, 'mod1')
+        mod1.write(code)
+        self._rename(mod1, code.index('a_var'), 'new_var', docs=True)
+        self.assertEquals('new_var = 1\n# new_var\n', mod1.read())
+
+    def test_not_renaming_in_strings_and_comments_where_not_visible(self):
+        code = 'def f():\n    a_var = 1\n# a_var\n'
+        mod1 = self.pycore.create_module(self.project.root, 'mod1')
+        mod1.write(code)
+        self._rename(mod1, code.index('a_var'), 'new_var', docs=True)
+        self.assertEquals('def f():\n    new_var = 1\n# a_var\n', mod1.read())
+
+    def test_not_renaming_all_text_occurrences_in_strings_and_comments(self):
+        code = 'a_var = 1\n# a_vard _a_var\n'
+        mod1 = self.pycore.create_module(self.project.root, 'mod1')
+        mod1.write(code)
+        self._rename(mod1, code.index('a_var'), 'new_var', docs=True)
+        self.assertEquals('new_var = 1\n# a_vard _a_var\n', mod1.read())
+
 
 class ChangeOccurrencesTest(unittest.TestCase):