Anonymous avatar Anonymous committed 8da1c92

Parse hints from cookies file. Implement exclude_paths hint.

Comments (0)

Files changed (3)

     toolshelf build Gettysburg-Address
 If there is an ambiguity, one such source will be picked non-deterministically
-(TODO: perhaps this should be an error?)  ♦ In which case, you may add the
-username and/or the site name to resolve the ambiguity.
+(TODO: perhaps this should be an error?)  In this case, you must add both the
+host name and the username to resolve the ambiguity:
+    toolshelf build
 ### How does it know which directories to place on your path? ###
 Example of an entry in the cookies file:
-      exclude_path tests
+      exclude_paths tests
       build_command ./configure --with-lighter-fluid --no-barbecue && make
 It should be possible to have a local cookies files that supplements
 The names of hints are as follows.
-*   ♦ `requires_executable`
+*   ♦ `requires_executables`
-    Example: `requires_executable perl`
+    Example: `requires_executables perl`
     A space-separated list of executables required to dock and run the source.
     When this is given, `toolshelf` first checks if you have the named
 *   ♦ `rectify_permissions`
-    Example: `rectify_permissions`
+    Example: `rectify_permissions yes`
     means rectify the execute permissions; after checking out the given
     source but before building it, traverse all of the files in the source
     whether `file` called it `executable` or not.  (This is the default for
     `.zip` archives.)
-*   ♦ `prerequisite`
+*   ♦ `prerequisites`
-    Example: `prerequisite gh:Scriptor/Pharen`
+    Example: `prerequisites gh:Scriptor/Pharen`
     indicates a dependency source tree.  When this is given, `toolshelf`
     first checks if you have the source named by the hint's value, a source
     specification, docked; if you do not, it will try to dock that source first.
-*   `exclude_path`
+*   `exclude_paths`
     indicates a directory subtree that you should not be added to the
     executable search path.  This could be useful if there are executables
     following directories from being put on the path: `tests/x86/passing`,
     `tests/x86/failing`, `tests/x8600`.
-*   ♦ `only_path`
+*   ♦ `only_paths`
-    Example: `only_path bin`
+    Example: `only_paths bin`
     indicates that *only* these subdirectories should be added to the
     executable search path.
   has no problem building it, finding the built executables, and putting them
   on your path.
-  ♦ `exclude_path scripts` is a hint specifier which says to decline putting
-  any paths from this project where the final directory is called `scripts`
-  on the search path.  This prevents the scripts that ship with `tenyr` from
-  being put on your path (because they have rather generic names, and are
-  probably not things that you would use frequently.)
+  In `toolshelf`'s cookies database, this source has the hint
+  `exclude_paths bench ui hw scripts` associated with it; it says to decline
+  putting any paths from this project which begin with `bench`, `ui`, `hw`,
+  or `scripts` onto the search path.  This prevents several scripts with
+  rather generic names, and which you would typically not use frequently, from
+  appearing on the search path.  These scripts can still be run by giving the
+  full path to them, of course.
 * `toolshelf dock `[``][]
   your `toolshelf` with the above command.  After it's docked, you can issue
   the commands `toolshelf path disable` and
   `toolshelf path rebuild` to remove or reinstate
-  it from your search path, respectively.
-  TODO ♦ `{x=tests:x=etc:x=examples:x=build-aux}`
+  it from your search path, respectively.  Similar to `tenyr`, this source has
+  the hint `exclude_paths tests etc examples build-aux` associated with it.
 # see if it is listed here.  If it is, it uses the hints listed in this file.
-  exclude_paths scripts
+  exclude_paths bench ui hw scripts
   exclude_paths tests
 def run(*args):
-    note("* Runnning `%s`..." % ' '.join(args))
+    note("Running `%s`..." % ' '.join(args))
 def chdir(path):
-    note("* Changing dir to `%s`..." % path)
+    note("Changing dir to `%s`..." % path)
 def note(msg):
     if OPTIONS.verbose:
-        print msg
+        print "*", msg
 def expand_docked_specs(specs, default_all=False):
         self.filename = os.path.join(
             TOOLSHELF, '.toolshelf', 'cookies.catalog'
-        self._specs = None
+        self._hint_map = None
-    def _load_catalog(self):
-        self._specs = []
+    def _load_hints(self):
+        self._hint_map = {}
         with open(self.filename, 'r') as file:
-            spec = {'hints': {}}
+            spec_key = None
             for line in file:
                 line = line.strip()
                 found_hint = False
                     pattern = r'^%s\s+(.*?)\s*$' % hint_name
                     match = re.match(pattern, line)
                     if match:
-                        spec['hints'][hint_name] =
+                        if spec_key is None:
+                            raise SourceSpecError(
+                                'Found hint %s before any spec' % hint_name
+                            )
+                        hint_value =
+                        note("Adding hint '%s %s' to %s" %
+                            (hint_name, hint_value, spec_key)
+                        )
+                        self._hint_map[spec_key][hint_name] = hint_value
                         found_hint = True
                 if found_hint or line == '' or line.startswith('#'):
-                spec['spec'] = parse_source_spec(line)
-                self._specs.append(spec)
-                spec = {'hints': {}}
-        self._specs.append(spec)
+                spec = parse_source_spec(line)
+                spec_key = os.path.join(
+                    spec['host'], spec['user'], spec['project']
+                )
+                self._hint_map.setdefault(spec_key, {})
-    def specs(self):
-        if self._specs is None:
-            self._load_catalog()
-        return self._specs
+    def hint_map(self):
+        if self._hint_map is None:
+            self._load_hints()
+        return self._hint_map
     def apply_hints(self, source):
-        #if in self.source_map:
-        #    source.hints = self.source_map[].hints
-        pass
+        source.hints.update(self.hint_map.get(, {}))
 class Path(object):
         self.user = user or 'distfile'
         self.project = project
         self.type = type
-        self.hints = ''
+        self.hints = {}
     def __repr__(self):
         return os.path.isdir(self.dir)
     def checkout(self):
-        note("* Checking out %s..." %
+        note("Checking out %s..." %
     def build(self):
         if not
-            note("* SKIPPING build of %s" %
+            note("SKIPPING build of %s" %
-        note("* Building %s..." % self.dir)
+        note("Building %s..." % self.dir)
         if os.path.isfile(''):
             raise NotImplementedError
     def may_use_path(self, dirname):
-        # TODO: rewrite this to use new hints
-        use_it = True
-        for hint in self.hints.split(':'):
-            # TODO: better hint parsing
-            try:
-                (name, value) = hint.split('=')
-            except ValueError:
-                continue
-            if name == 'x':
-                verboten = os.path.join(self.dir, value)
-                if dirname.startswith(verboten):
-                    use_it = False
-                    break
-        return use_it
+        exclude_paths = self.hints.get('exclude_paths', '').split(' ')
+        if exclude_paths == ['']:
+            return True
+        for path in exclude_paths:
+            verboten = os.path.join(self.dir, path)
+            if dirname.startswith(verboten):
+                return False
+        return True
     def find_path_components(self):
         index = {}
         components = []
         for dirname in sorted(index):
             if not self.may_use_path(dirname):
-                note("(SKIPPING %s)" % dirname)
+                note("%s excluded from search path" % dirname)
             note("  %s:" % dirname)
             for filename in index[dirname]:
                     if 'executable' in output:
                         make_it_executable = True
                     if make_it_executable:
-                        note("* Making %s executable" % name)
+                        note("Making %s executable" % name)
                         subprocess.check_call(["chmod", "u+x", filename])
-                        note("* Making %s NON-executable" % name)
+                        note("Making %s NON-executable" % name)
                         subprocess.check_call(["chmod", "u-x", filename])
     for source in sources:
-    # XXX overkill for now.  should be like
-    # + [ for s in sources]
-    # except s.spec, or make parseable as a spec
-    path_cmd(result, ['rebuild', 'all'])
+    path_cmd(result, ['rebuild'] + [ for s in sources])
 def path_cmd(result, args):
         # special case to handle total rebuilds/disables:
         # XXX all= no longer works, fix this somehow
         if all:
-            note("* Removing from your PATH all toolshelf directories")
+            note("Removing from your PATH all toolshelf directories")
-            note("* Removing from your PATH all directories that "
+            note("Removing from your PATH all directories that "
                  "start with one of the following...")
             for source in sources:
                 note("  " + source.dir)
             raise SourceSpecError(repr(problems))
         p = Path()
         clean_path(p, sources, all=(specs == ['all']))
-        note("* Adding the following executables to your PATH...")
+        note("Adding the following executables to your PATH...")
         for source in sources:
             for component in source.find_path_components():
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.