Source

MyOhData / myohdata.py

Full commit
"""\
========
MyOhData
========


Overview
--------

This is a simple OData server using CherryPy and Jinja2. It doesn't support much
of the OData_ spec. It's basically the result of me trying to wrap my head 
around OData_ and how to provide it.


Installation
------------

Run following commands at a terminal (I suggest doing this in a virtualenv_)::

    $ tar xzvf myohdata-1.0.tar.gz
    $ cd myohdata-1.0/
    $ easy_install "CherryPy >= 3.2" Jinja2
    $ python myohdata.py

That will launch an OData_ service running on `localhost:8011` with some sample
data.


OData Features
--------------

This implementation is rather slim on features. You can host multiple catalogs,
list the entries in catalogs and access individual entries. That's it. It
supports a subset of the Atom version of OData. It is read-only.

The ``/``, ``/$metadata``, ``/<Catalog>`` and ``/<Catalog>(<id>)`` endpoints
have been validated against http://services.odata.org/validation/ and
conform to all of their rules as of 2011-04-22.


Advanced
--------

Here are some details in case you want to play with more than just the one
sample catalog. The data source that the app reads from is a Python dictionary.
The dictionary represents `catalogs` of `entries`. 


Catalogs
~~~~~~~~

Catalogs should be named with strings. Here is an example::

    all_data = {'Widgets':{ ... }}

One catalog named "Widgets" is defined there.  What should a catalog contain,
you ask? Metadata and entries is the answer!


Catalog Metadata
~~~~~~~~~~~~~~~~

The catalog metadata describes the content of the catalog - namely, the
structure of the entities that it contains. It is a sub-dictionary off of the
catalog and must be keyed with `_meta`. Here are the required fields.

`key`

    `str` - The field in each entity that represents the unique key that
    identifies the entity.

`title`

    `str` - The field in each entity that represents the display title of the
    entity.

`name`

    `str` - The name of the entity. Usually singular -- `Widget`, for example.

`set`

    `str` - The name of a set of the entities. Usually plural -- `Widgets`. You
    get the idea.

`properties`

    `list of tuples` - Each tuple in this list represents a property on the
    entity. The tuples should contain three values.

    1. `str` The name of the property.
    2. `str` The `EDM type`_ of the property's value.
    3. `str` Whether the property can be null -- either "true" or "false".

.. _`metadata example`:

Here is an example catalog with metadata (no entries).::

    {'Widgets':
        '_meta': {
            'key':'ID', 
            'name':'Widget',
            'set':'Widgets',
            'title':'Name',
            'properties':[
                ('ID', 'Edm.Int32', 'false'),
                ('Name', 'Edm.String', 'false'),
                ('Function', 'Edm.String', 'true'),
            ],
        },
    }


Catalog Entries
~~~~~~~~~~~~~~~

Conceptually, an entry is an instance of an entity.  Every other key/value pair
in the catalog must represent an entry.  The `key` must be equal to
``value[meta['key']]``. The `value` should be a dictionary that maps to the
properties defined for the entity. Here is an example of a catalog of entries
that corresponds to the `metadata example`_ above::

    {'Widgets':
        ...
        1: {'ID':1, 'Name':'Foo', 'Function':'Doing foo-like things.'},
        2: {'ID':2, 'Name':'Bar', 'Function':'Processing bars.'},
        3: {'ID':3, 'Name':'Baz', 'Function':'BAAAAZ!.'},
        ...
    }


Complete Catalog Example
~~~~~~~~~~~~~~~~~~~~~~~~

Putting it all together, we get::

    {'Widgets':

        '_meta': {
            'key':'ID', 
            'name':'Widget',
            'set':'Widgets',
            'title':'Name',
            'properties':[
                ('ID', 'Edm.Int32', 'false'),
                ('Name', 'Edm.String', 'false'),
                ('Function', 'Edm.String', 'true'),
            ],
        },

        1: {'ID':1, 'Name':'Foo', 'Function':'Doing foo-like things.'},
        2: {'ID':2, 'Name':'Bar', 'Function':'Processing bars.'},
        3: {'ID':3, 'Name':'Baz', 'Function':'BAAAAZ!.'},
    }


TODO
----

I doubt that I'll add these features, but the following would be nice:

* `Query options`_
* Associations_
* Full CRUD operations
* Support for paging large catalogs


Contact
-------

I'm Christian Wyglendowski. I work for YouGov_. You can find me on the web on `my
site`_, Twitter_ or shoot me an email (christian@dowski.com).

.. _OData: http://www.odata.org/
.. _`EDM type`: http://www.odata.org/developers/protocols/overview#AbstractTypeSystem
.. _`Query options`: http://www.odata.org/developers/protocols/uri-conventions#QueryStringOptions
.. _Associations: http://www.odata.org/developers/protocols/overview#EntityDataModel
.. _virtualenv: http://www.virtualenv.org/
.. _`my site`: http://www.dowski.com/
.. _Twitter: https://twitter.com/#!/dowskimania
.. _YouGov: http://today.yougov.com/

"""
import datetime
import re

from textwrap import dedent
from urlparse import urlparse

import cherrypy

from jinja2 import Template


__author__ = "Christian Wyglendowski"
__license__ = "MIT"


class XMLObject(object):
    """Base class that exposes a resource and sets some headers."""

    exposed = True
    _cp_config = {
        'tools.response_headers.on':True,
        'tools.response_headers.headers': [
            ('Content-Type', 'application/xml'),
            ('DataServiceVersion', '2.0'),
        ]
    }


class MyOhDataRoot(XMLObject):
    """The OData service root. Returns a list of configured catalogs."""

    def __init__(self, data):
        self.data = data
        self.template = Template(open('root.xml').read())
        self.catalog_metadata = []
        for name, catalog in data.iteritems():
            meta = catalog.pop('_meta', {})
            setattr(self, name, MyOhDataFeed(name, catalog, meta))
            self.catalog_metadata.append(meta)
        self._metadata = MyOhDataServiceMetadata(self.catalog_metadata)

    def GET(self):
        return self.template.render(
            catalog_metadata=self.catalog_metadata,
            netloc=urlparse(cherrypy.url()).netloc,
        )

class MyOhDataFeed(XMLObject):
    """An OData feed. Returns the whole feed or single entries."""

    def __init__(self, name, catalog, metadata):
        self.name = name
        self.data = catalog
        self.meta = metadata
        self.feed_template = Template(open('catalog.xml').read())
        self.entry_template = Template(open('entry.xml').read())
        self._set_timestamp()

    def GET(self, item=None, **params):
        cherrypy.response.headers['Content-Type'] = 'application/atom+xml'
        fill = {
            'netloc': urlparse(cherrypy.url()).netloc,
            'count': len(self.data),
            'meta':self.meta,
            'timestamp':self.timestamp
        }
        if not item:
            fill['catalog'] = self.data.iteritems()
            return self.feed_template.render(**fill)
        else:
            try:
                fill['entry'] = self.data[int(item)]
            except KeyError:
                raise cherrypy.NotFound()
            return self.entry_template.render(**fill)

    def _set_timestamp(self):
        """All documents will have a last updated time of server-start."""
        utcnow = datetime.datetime.utcnow()
        self.timestamp = utcnow.strftime('%Y-%m-%dT%H:%M:%SZ')

class MyOhDataServiceMetadata(XMLObject):
    """OData Service Metadata document."""

    def __init__(self, catalog_metadata):
        self.catalog_metadata = catalog_metadata
        self.template = Template(open('metadata.xml').read())

    def GET(self):
        return self.template.render(catalog_metadata=self.catalog_metadata)

class ODataDispatcher(cherrypy.dispatch.MethodDispatcher):
    """A dispatcher that handles OData /Catalog(item) style URLs."""

    pattern = re.compile(r'(/[a-zA-Z][a-zA-Z0-9_]*)(?:\((\d+)\))?')
    def find_handler(self, path_info):
        item = None
        match = self.pattern.match(path_info)
        if match:
            catalog, item = match.groups()
            if item:
                path_info = catalog
        handler, vpath = super(ODataDispatcher, self).find_handler(path_info)
        if vpath:
            handler = None
        return handler, [item] if item else vpath

def main(data):
    """Starts the service given a data dictionary with catalogs in it."""
    configuration = {
        '/': {
            'request.dispatch':ODataDispatcher(),
        },
    }

    cherrypy.config.update({
        'server.socket_host':'0.0.0.0',
        'server.socket_port':8011,
    })

    cherrypy.quickstart(MyOhDataRoot(data), config=configuration)


if __name__ == '__main__':

    # Sample data.
    data = {
        'Widgets': {

            # Metadata that describes the entries and the set.
            # ------------------------------------------------
            '_meta': {
                'key':'ID', 
                'name':'Widget',
                'set':'Widgets',
                'title':'Name',
                'properties':[
                    ('ID', 'Edm.Int32', 'false'),
                    ('Name', 'Edm.String', 'false'),
                    ('Function', 'Edm.String', 'true'),
                ],
            },

            # The entries themselves.
            # -----------------------
            1: {'ID':1, 'Name':'Foo', 'Function':'Doing foo-like things.'},
            2: {'ID':2, 'Name':'Bar', 'Function':'Processing bars.'},
            3: {'ID':3, 'Name':'Baz', 'Function':'BAAAAZ!.'},
            5: {'ID':4, 'Name':'FooBar', 'Function':'It\'s foo-meets-bar.'},
        },
    }

    main(data)