Source

slagg /

Filename Size Date modified Message
examples
slagg
36 B
4.0 KB
222 B
362 B
.. -*- restructuredtext -*-

============
slagg Readme
============

About
-----

slagg is a high-performance aggregator library for `Google App Engine`_.
Aggregations are performed using tasks.

.. _Google App Engine: http://code.google.com/appengine


    Please Note: Some minor API changes are very likely to occur soon.


Installing
----------

Copy or link the ``slagg`` directory to your application's root directory.


Basic Usage
-----------

The basic `slagg` implementation will calculate a sum.  To customize behavior,
subclass one of the `slagg` handlers and override the desired settings or methods.

`slagg` consists of three primary handlers:

FanoutHandler
    Used to implement 'reversal' logic and to 'bypass' the limit on transactional tasks.

CreateWorkHandler
    Creates a 'work-model' and tries to insert an aggregator task.

AggregationHander
    Process the work-models, then updates the aggregate entity.

Your application must define the aggregate model.  The default implementation
expects your aggregate to have a value attribute, which should be an 
IntegerProperty and default to 0.

Code is often best explained via example::

    import wsgiref.handlers
    from google.appengine.ext import webapp
    from google.appengine.ext import db

    import slagg

    # Must be defined by your app
    class TestModel(db.Model):
        rev = db.IntegerProperty(default=0)
        value = db.IntegerProperty()

    class AggModel(db.Model):
        value = db.IntegerProperty(default=0)

    # subclassed to customize settings
    class CreateWorkTestHandler(slagg.CreateWorkHandler):
        delay = 15
        queue_name = 'default'
        creatework_url = '/slagg/creatework'
        base_url = '/slagg/aggregate/'

    # must be subclassed to specify the AggModel
    class AggregationTestHandler(slagg.AggregationHandler):
        AggModel = AggModel

    # then just call your CreateWorkHandler's create_work method.
    class SaveModelHandler(webapp.RequestHandler):
        def get(self):
            import random
            def txn():
                entity = TestModel()
                entity.rev += 1
                entity.value = int(random.random() * 500)
                entity_key = entity.put()
                CreateWorkTestHandler.create_work(aggregation_url='aggregate',
                                                  entity_id=entity_key.id(),
                                                  entity_revision=entity.rev,
                                                  state=entity.value)
            db.run_in_transaction(txn)
            self.response.out.write('New TestModel created.')


    def main():
      application = webapp.WSGIApplication([('/slagg', SaveModelHandler),
                                            ('/slagg/creatework', CreateWorkTestHandler),
                                            ('/slagg/aggregate/*', AggregationTestHandler)],
                                           debug=True)
      wsgiref.handlers.CGIHandler().run(application)
    
    if __name__ == '__main__':
      main()

For a more complete example see `basic_usage.py` in the examples directory.  Copy
`basic_usage.py` to your app's root directory and add a `slagg` section to your app.yaml::

    - url: /slagg.*
      script: test_slagg.py
      login: required


Now visit the following URLs to watch `slagg` in action:

http://localhost:8080/slagg
    Runs a handler that automatically generates a random number of entities.
    Run repeatedly and watch your task queue and the datastore (AggWork, AggModel,
    and AggState) to see how `slagg` works behind the scenes.  This example is
    contrived purely so you can observe how the aggregation is runs with *many*
    updates.

http://localhost:8080/slagg/one
    This example is a much more real-world usage example.  Because it is called
    from within a transaction, create_work is called transactionaly.

http://localhost:8080/slagg/verify
    Check this page as you watch the tasks run.  This lets you make sure the
    code is at least minimally sane.  :)