jasonjwwilliams avatar jasonjwwilliams committed e5359dc

Added key filter support for map/reduce jobs.

* RiakClient.add() and RiakMapReduce.add() now support the
passing of key filters. You specify a key filter as input
by passing a dictionary of the format:

{"bucket" : "<bucket_name>",
"key_filters" : [ ["<operation>", "<arg1>", "<argN>"],

e.g (key ID in form "log_web_20110302174601"):

{"bucket" : "my_cool_bucket",
"key_filters" : [ ["tokenize", "_", "2"],
["eq", "web"]]}

Read the docs on key filters for more info on the
commands: https://wiki.basho.com/Key-Filters.html

Comments (0)

Files changed (2)

         different forms, depending on the provided inputs. You can
         specify either a RiakObject, a string bucket name, or a bucket,
         key, and additional arg.
-        :param arg1: RiakObject or Bucket
+        :param arg1: RiakObject or Bucket or Key Filter
         :param arg2: Key or blank
         :param arg3: Arg or blank
         :returns: RiakMapReduce
+        Key filters are dictionaries of the form:
+            {"bucket" : "<bucket_name>",
+             "key_filters" : [ ["<operation>", "<arg1>", "<argN>"],
+                             ]}
         if (arg2 == None) and (arg3 == None):
             if isinstance(arg1, RiakObject):
                 return self.add_object(arg1)
+            elif isinstance(arg1, dict):
+                return self.add_key_filter(arg1)
                 return self.add_bucket(arg1)
     def add_bucket_key_data(self, bucket, key, data):
         """Add data to the bucket."""
-        if self._input_mode == 'bucket':
-            raise Exception('Already added a bucket, can\'t add an object.')
+        if self._input_mode == 'bucket' or \
+           self._input_mode == 'key_filter':
+            raise Exception(('Already added a %s, ' % self._input_mode) +
+                            'can\'t add an object.')
             self._inputs.append([bucket, key, data])
             return self
         self._input_mode = 'bucket'
         self._inputs = bucket
         return self
+    def add_key_filter(self, key_filter):
+        """Set up key filter to use."""
+        if self._input_mode == 'bucket':
+            raise Exception('Already added a bucket can\'t add a key filter.')
+        else:
+            self._input_mode = "key_filter"
+            self._inputs = key_filter
+            return self
     def link(self, bucket='_', tag='_', keep=False):
     def tearDown(self):
         """delete all the bucket objects we might be using"""
-        keys = ['foo', 'foo1', 'foo2', 'bar', 'baz',
+        keys = ['foo', 'foo1', 'foo2', 'bar', 'baz', 'ba_foo1'
                 'foo1', 'foo2', 'foo3', 'blue_foo1']
         yield cleanup_bucket(keys)
         self.assertEqual(result, [9])
         log.msg('done javascript_named_map_reduce')
+    @defer.inlineCallbacks
+    def test_javascript_key_filter_map_reduce(self):
+        """javascript map/reduce using key filters"""
+        log.msg("javascript map reduce with key filter")
+        client = riak.RiakClient(client_id=RIAK_CLIENT_ID)
+        bucket = client.bucket("bucket")
+        yield bucket.new("foo", 2).store()
+        yield bucket.new("bar", 3).store()
+        yield bucket.new("baz", 4).store()
+        # Run the map...
+        job = client \
+                .add({"bucket" : "bucket",
+                      "key_filters" : [["starts_with", "ba"]]}) \
+                .map("function (v) { return [1]; }") \
+                .reduce("function(v) { if(v.length) return [v.length]; else return []} ")
+        result = yield job.run()
+        self.assertEqual(result, [2])
+        log.msg('done javascript_key_filter_map_reduce')
     def test_javascript_bucket_map_reduce(self):
         """javascript bucket map reduce"""
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.