Anonymous avatar Anonymous committed b91b2ce

docs

Comments (0)

Files changed (5)

docs/build/api.rst

 .. automodule:: dogpile.cache.region
     :members:
 
+.. autofunction:: dogpile.cache.util.function_key_generator
+
 Backend API
 =============
 

docs/build/front.rst

+============
+Front Matter
+============
+
+Information about the dogpile.cache project.
+
+Project Homepage
+================
+
+dogpile.cache is hosted on `Bitbucket <http://bitbucket.org>`_ - the lead project page is at https://bitbucket.org/zzzeek/dogpile.cache.  Source
+code is tracked here using `Mercurial <http://mercurial.selenic.com/>`_.
+
+Releases and project status are available on Pypi at http://pypi.python.org/pypi/dogpile.cache.
+
+The most recent published version of this documentation should be at http://packages.python.org/dogpile.cache/.
+
+Installation
+============
+
+Install released versions of dogpile.cache from the Python package index with `pip <http://pypi.python.org/pypi/pip>`_ or a similar tool::
+
+    pip install dogpile.cache
+
+Installation via source distribution is via the ``setup.py`` script::
+
+    python setup.py install
+
+Community
+=========
+
+dogpile.cache is developed by `Mike Bayer <http://techspot.zzzeek.org>`_, and is 
+loosely associated with the `Pylons Project <http://www.pylonsproject.org/>`_.
+As dogpile.cache's usage increases, it is anticipated that the Pylons mailing list and IRC channel
+will become the primary channels for support.
+
+Bugs
+====
+
+Bugs and feature enhancements to dogpile.cache should be reported on the `Bitbucket
+issue tracker
+<https://bitbucket.org/zzzeek/dogpile.cache/issues?status=new&status=open>`_.   If you're not sure 
+that a particular issue is specific to either dogpile.cache or `Dogpile <https://bitbucket.org/zzzeek/dogpile>`_, posting to the dogpile.cache
+tracker is likely the better place to post first.
+
+* `dogpile.cache issue tracker <https://bitbucket.org/zzzeek/dogpile.cache/issues?status=new&status=open>`_ (post here if unsure)
+* `dogpile issue tracker <https://bitbucket.org/zzzeek/dogpile/issues?status=new&status=open>`_
+

docs/build/index.rst

 `dogpile.cache <http://bitbucket.org/zzzeek/dogpile.cache>`_ provides a simple 
 caching pattern based on the `dogpile <http://pypi.python.org/pypi/dogpile>`_
 locking system, including rudimentary backends. It effectively completes the
-replacement of Beaker as far as caching is concerned, providing an open-ended
-and simple pattern to configure caching. New backends are very easy to create
+replacement of `Beaker <http://beaker.groovie.org>`_ as far as caching (though **not** HTTP sessions) 
+is concerned, providing an open-ended, simple, and higher-performing pattern to configure and use 
+cache backends. New backends are very easy to create
 and use; users are encouraged to adapt the provided backends for their own
 needs, as high volume caching requires lots of tweaks and adjustments specific
 to an application and its environment.
 .. toctree::
    :maxdepth: 2
 
+   front
    usage
    api
 

docs/build/usage.rst

+============
+Usage Guide
+============
+
 .. note:: dogpile.cache is **not released or completed** at this time.   Development
    is currently in progress and the current code is not yet functional.
 
-Introduction
-============
+Overview
+========
 
-At its core, Dogpile provides a locking interface around a "value creation" function.
+At the time of this writing, popular key/value servers include 
+`Memcached <http://memcached.org>`_, `Redis <http://redis.io/>`_, and `Riak <http://wiki.basho.com/>`_.
+While these tools all have different usage focuses, they all have in common that the storage model
+is based on the retrieval of a value based on a key; as such, they are all potentially
+suitable for caching, particularly Memcached which is first and foremost designed for
+caching.   
 
-The interface supports several levels of usage, starting from
-one that is very rudimentary, then providing more intricate 
-usage patterns to deal with certain scenarios.  The documentation here will attempt to 
-provide examples that use successively more and more of these features, as 
-we approach how a fully featured caching system might be constructed around
-Dogpile.
-
-Note that when using the `dogpile.cache <http://bitbucket.org/zzzeek/dogpile.cache>`_
-package, the constructs here provide the internal implementation for that system,
-and users of that system don't need to access these APIs directly (though understanding
-the general patterns is a terrific idea in any case).
-Using the core Dogpile APIs described here directly implies you're building your own 
-resource-usage system outside, or in addition to, the one 
-`dogpile.cache <http://bitbucket.org/zzzeek/dogpile.cache>`_ provides.
-
-Usage
-=====
+With a caching system in mind, dogpile.cache provides an interface to a particular Python API
+targeted at that system.  In this document, we'll illustrate Memcached usage 
+using the `pylibmc <http://pypi.python.org/pypi/pylibmc>`_ backend, which is a high performing 
+Python library for Memcached.  It can be compared to the `python-memcached <http://pypi.python.org/pypi/python-memcached>`_
+client, which is also an excellent product.  Pylibmc is written against Memcached's native API
+so is markedly faster, though might be considered to have rougher edges.   The API is actually a bit 
+more verbose to allow for correct multithreaded usage.
 
 A dogpile.cache configuration consists of the following components:
 
-* A *region*, which is an instance of ``CacheRegion``, and defines the configuration
-  details for a particular cache backend.
-* A *backend*, which is an instance of ``CacheBackend``, describing how values
+* A *region*, which is an instance of :class:`.CacheRegion`, and defines the configuration
+  details for a particular cache backend.  The :class:`.CacheRegion` can be considered
+  the "front end" used by applications.
+* A *backend*, which is an instance of :class:`.CacheBackend`, describing how values
   are stored and retrieved from a backend.  This interface specifies only
-  ``get()``, ``put()`` and ``delete()``.
+  :meth:`~.CacheBackend.get`, :meth:`~.CacheBackend.put` and :meth:`~.CacheBackend.delete`.
+  The actual kind of :class:`.CacheBackend` in use for a particular :class:`.CacheRegion`
+  is determined by the underlying Python API being used to talk to the cache, such
+  as Pylibmc.  The :class:`.CacheBackend` is instantiated behind the scenes and 
+  not directly accessed by applications under normal circumstances.
 * Value generation functions.   These are user-defined functions that generate
-  new values to be placed in the cache.
+  new values to be placed in the cache.   While dogpile.cache offers the usual
+  "put" approach of placing data into the cache, the usual mode of usage is to only instruct
+  it to "get" a value, passing it a *creation function* which will be used to 
+  generate a new value if and only if one is needed.   This "get-or-create" pattern
+  is the entire key to the "Dogpile" system, which coordinates a single value creation 
+  operation among many concurrent get operations for a particular key, eliminating
+  the issue of an expired value being redundantly re-generated by many workers simultaneously.
 
-The most common caching style in use these days is via memcached, so an example
-of this using the `pylibmc <http://pypi.python.org/pypi/pylibmc>`_ backend looks like::
+Rudimentary Usage
+=================
+
+dogpile.cache includes a Pylibmc backend.  A basic configuration looks like::
 
     from dogpile.cache import make_region
 
     def load_user_info(user_id):
         return some_database.lookup_user_by_id(user_id)
 
-Above, we create a ``CacheRegion`` using the ``make_region()`` function, then
-apply the backend configuration via the ``configure()`` method, which returns the 
+Above, we create a :class:`.CacheRegion` using the :func:`.make_region` function, then
+apply the backend configuration via the :meth:`.CacheRegion.configure` method, which returns the 
 region.  The name of the backend is the only required argument,
 in this case ``dogpile.cache.pylibmc``.
 
-Subsequent arguments then include *expiration_time*, which is the expiration 
+The configuration is separated into two sections.  Upon construction via :func:`.make_region`,
+the :class:`.CacheRegion` object is available, typically at module
+import time, for usage in decorating functions.   Additional configuration details passed to
+:meth:`.CacheRegion.configure` are typically loaded from a configuration file and therefore
+not necessarily available until runtime, hence the two-step configurational process.
+
+Key arguments passed to :meth:`.CacheRegion.configure` include *expiration_time*, which is the expiration 
 time passed to the Dogpile lock, and *arguments*, which are arguments used directly
 by the backend - in this case we are using arguments that are passed directly
 to the pylibmc module.
 
+Region Configuration
+====================
+
+The :func:`.make_region` function currently calls the :class:`.CacheRegion` constructor directly.
+
+.. autoclass:: dogpile.cache.region.CacheRegion
+    :noindex:
+
+One you have a :class:`.CacheRegion`, the :meth:`.CacheRegion.cache_on_arguments` method can
+be used to decorate functions, but the cache itself can't be used until
+:meth:`.CacheRegion.configure` is called.  The interface for that method is as follows:
+
+.. automethod:: dogpile.cache.region.CacheRegion.configure
+    :noindex:
+
+The :class:`.CacheRegion` can also be configured from a dictionary, using the :meth:`.CacheRegion.configure_from_config`
+method:
+
+.. automethod:: dogpile.cache.region.CacheRegion.configure_from_config
+    :noindex:
+
+
+Using a Region
+==============
+
+The :class:`.CacheRegion` object is our front-end interface to a cache.  It includes
+the following methods:
+
+
+.. automethod:: dogpile.cache.region.CacheRegion.get
+    :noindex:
+
+.. automethod:: dogpile.cache.region.CacheRegion.get_or_create
+    :noindex:
+
+.. automethod:: dogpile.cache.region.CacheRegion.put
+    :noindex:
+
+.. automethod:: dogpile.cache.region.CacheRegion.delete
+    :noindex:
+
+.. automethod:: dogpile.cache.region.CacheRegion.cache_on_arguments
+    :noindex:
+
 Backends
 ========
 
 backend in the same way as if it were part of the existing sys.path.
 
 For example, to create a backend called ``DictionaryBackend``, we subclass
-``CacheBackend``::
+:class:`.CacheBackend`::
 
     from dogpile.cache import CacheBackend, NO_VALUE
 
 pickle or similar can be used on the tuple - the "metadata" portion will always
 be a small and easily serializable Python structure.
 
-Region Arguments
-================
-
-The ``make_region()`` function accepts these arguments:
-
-``name``
-
-  Optional.  A string name for the region.  This isn't used internally
-  but can be accessed via the ``.name`` parameter, helpful
-  for configuring a region from a config file.
-
-``function_key_generator``
-
-  Optional.  Plug in a function that will produce a "cache key" given 
-  a data creation function and arguments.   The structure of this function
-  should be two levels: given the data creation function, return a new
-  function that generates the key based on the given arguments.  Such
-  as::
-
-    def my_key_generator(fn):
-        namespace = fn.__name__
-        def generate_key(*arg):
-            return namespace + "_".join(str(s) for s in arg)
-        return generate_key
-
-
-    region = make_region(
-        function_key_generator = my_key_generator
-    ).configure(
-        "dogpile.cache.dbm",
-        expiration_time=300,
-        arguments={
-            "filename":"file.dbm"
-        }
-    )
-
-``key_mangler``
-
-  Optional.  Function which will "mangle" the incoming keys.  If left
-  at ``None``, the backend may provide a default "mangler" function.
-  Set to ``False`` to unconditionally disable key mangling.
-
-One you have a ``CacheRegion``, the ``cache_on_arguments()`` method can
-be used to decorate functions, but the cache itself can't be used until
-``configure()`` is called.  That method accepts these arguments:
-
-``backend``
-  Required.  This is the name of the ``CacheBackend`` to use, and
-  is resolved by loading the class from the ``dogpile.cache`` entrypoint.
-
-``expiration_time``
-
-  Optional.  The expiration time passed to the dogpile system.  The ``get_or_create()``
-  method as well as the ``cache_on_arguments()`` decorator (note:  **not** the
-  ``get()`` method) will call upon the value creation function after this
-  time period has passed since the last generation.
-
-``arguments``
-
-  Optional.  The structure here is passed directly to the constructor
-  of the ``CacheBackend`` in use, though is typically a dictionary.
-
-Configure Region from a Configuration Dictionary
-================================================
-
-Call ``configure_from_config()`` instead::
-
-    local_region = make_region()
-    memcached_region = make_region()
-
-    # regions are ready to use for function
-    # decorators, but not yet for actual caching
-
-    # later, when config is available
-    myconfig = {
-        "cache.local.backend":"dogpile.cache.dbm",
-        "cache.local.arguments.filename":"/path/to/dbmfile.dbm",
-        "cache.memcached.backend":"dogpile.cache.pylibmc",
-        "cache.memcached.arguments.url":"127.0.0.1, 10.0.0.1",
-    }
-    local_region.configure_from_config(myconfig, "cache.local.")
-    memcached_region.configure_from_config(myconfig, "cache.memcached.")
-
-Using a Region
-==============
-
-The ``CacheRegion`` object is our front-end interface to a cache.  It includes
-the following methods:
-
-``get(key)``
-
-  Return a value from the cache, based on the given key.  While it's typical
-  the key is a string, it's passed through to the underlying backend so can
-  be of any type recognized by the backend.  If the value is not present, returns the 
-  token ``NO_VALUE``.  ``NO_VALUE`` evaluates to False, but is separate
-  from ``None`` to distinguish between a cached value of ``None``.
-  Note that the ``expiration_time`` argument is **not** used here - this method
-  is a direct line to the backend's behavior.
-
-``get_or_create(key, creator)``
-
-  Similar to ``get``, will use the given "creation" function to create a new
-  value if the value does not exist.   This will use the underlying dogpile/
-  expiration mechanism to determine when/how the creation function is called.
-
-``put(key, value)``
-
-  Place a new value in the cache under the given key.
-
-``delete(key)``
-
-  Remove a value from the cache.   This operation is idempotent (can be
-  called multiple times, or on a non-existent key, safely)
-
-``cache_on_arguments(fn)``
-
-  A function decorator that will cache the return value of the function
-  using a key derived from the name of the function, its location within
-  the application (i.e. source filename) as well as the arguments
-  passed to the function.
-
-  The generation of the key from the function is the big 
-  controversial thing that was a source of user issues with Beaker.  Dogpile
-  provides the latest and greatest algorithm used by Beaker, but also
-  allows you to use whatever function you want, by specifying it
-  to ``make_region()`` using the ``function_key_generator`` argument.
-
-
 Mako Integration
 ================
 

dogpile/cache/region.py

 """
 
 class CacheRegion(object):
-    """A front end to a particular cache backend."""
+    """A front end to a particular cache backend.
+    
+    :param name: Optional, a string name for the region.
+     This isn't used internally
+     but can be accessed via the ``.name`` parameter, helpful
+     for configuring a region from a config file.
+    :param function_key_generator:  Optional.  A 
+     function that will produce a "cache key" given 
+     a data creation function and arguments, when using
+     the :meth:`.CacheRegion.cache_on_arguments` method.
+     The structure of this function
+     should be two levels: given the data creation function, 
+     return a new function that generates the key based on 
+     the given arguments.  Such as::
+
+        def my_key_generator(fn):
+            namespace = fn.__name__
+            def generate_key(*arg):
+                return namespace + "_".join(str(s) for s in arg)
+            return generate_key
+
+
+        region = make_region(
+            function_key_generator = my_key_generator
+        ).configure(
+            "dogpile.cache.dbm",
+            expiration_time=300,
+            arguments={
+                "filename":"file.dbm"
+            }
+        )
+
+    :param key_mangler: Function which will be used on all incoming
+     keys before passing to the backend.  Defaults to ``None``,
+     in which case the key mangling function recommended by
+     the cache backend will be used.    A typical mangler
+     is the SHA1 mangler found at found at :meth:`.sha1_mangle_key` 
+     which coerces keys into a SHA1
+     hash, so that the string length is fixed.  To
+     disable all key mangling, set to ``False``.
+    :param lock_generator: Function which, given a cache key,
+     returns a mutexing object.
+    
+    """
 
     def __init__(self,
             name=None, 
             key_mangler=None
 
     ):
-        """Construct a new :class:`.CacheRegion`.
-        
-        :param name: Optional, name for the region.
-        :function_key_generator: Optional, key generator used by
-         :meth:`.CacheRegion.cache_on_arguments`.
-        :param key_mangler: Function which will be used on all incoming
-         keys before passing to the backend.  Defaults to ``None``,
-         in which case the key mangling function recommended by
-         the cache backend will be used.    A typical mangler
-         is the SHA1 mangler found at found at :meth:`.sha1_mangle_key` 
-         which coerces keys into a SHA1
-         hash, so that the string length is fixed.  To
-         disable all key mangling, set to ``False``.
-        :param lock_generator: Function which, given a cache key,
-         returns a mutexing object.
-        """
+        """Construct a new :class:`.CacheRegion`."""
         self.function_key_generator = function_key_generator
         if key_mangler:
             self.key_mangler = key_mangler
             _config_argument_dict=None,
             _config_prefix=None
         ):
-        """Configure a :class:`.CacheRegion`
+        """Configure a :class:`.CacheRegion`.
         
         The :class:`.CacheRegion` itself 
-        is returned..
+        is returned.
         
-        :param backend: Cache backend name
-        :param expiration_time: Expiration time, in seconds
-        :param arguments: Argument structure passed to the 
-         backend.  Is typically a dict.
+        :param backend:   Required.  This is the name of the 
+         :class:`.CacheBackend` to use, and is resolved by loading 
+         the class from the ``dogpile.cache`` entrypoint.
+
+        :param expiration_time:   Optional.  The expiration time passed 
+         to the dogpile system.  The :meth:`.CacheRegion.get_or_create`
+         method as well as the :meth:`.CacheRegion.cache_on_arguments` 
+         decorator (though note:  **not** the :meth:`.CacheRegion.get` method)
+         will call upon the value creation function after this
+         time period has passed since the last generation.
+
+        :param arguments:   Optional.  The structure here is passed directly 
+         to the constructor of the :class:`.CacheBackend` in use, though 
+         is typically a dictionary.
          
         """
         if "backend" in self.__dict__:
 
     def configure_from_config(self, config_dict, prefix):
         """Configure from a configuration dictionary 
-        and a prefix."""
+        and a prefix.
+        
+        Example::
+        
+            local_region = make_region()
+            memcached_region = make_region()
+
+            # regions are ready to use for function
+            # decorators, but not yet for actual caching
+
+            # later, when config is available
+            myconfig = {
+                "cache.local.backend":"dogpile.cache.dbm",
+                "cache.local.arguments.filename":"/path/to/dbmfile.dbm",
+                "cache.memcached.backend":"dogpile.cache.pylibmc",
+                "cache.memcached.arguments.url":"127.0.0.1, 10.0.0.1",
+            }
+            local_region.configure_from_config(myconfig, "cache.local.")
+            memcached_region.configure_from_config(myconfig, "cache.memcached.")
+
+        """
         return self.configure(
             config_dict["%s.backend" % prefix],
             expiration_time = config_dict.get("%s.expiration_time" % prefix, None),
         a particular value::
         
             generate_something.invalidate(5, 6)
-        
-        The mechanism used to generate cache keys is controlled
-        by the ``function_key_generator`` function passed
-        to :class:`.CacheRegion`. It defaults to :func:`.function_key_generator`.    
+
+        The generation of the key from the function is the big 
+        controversial thing that was a source of user issues with Beaker.  Dogpile
+        provides the latest and greatest algorithm used by Beaker, but also
+        allows you to use whatever function you want, by specifying it
+        to using the ``function_key_generator`` argument to :func:`.make_region`
+        and/or :class:`.CacheRegion`.  If defaults to :func:`.function_key_generator`.
 
         """
         key_generator = self.function_key_generator(fn)
 
         return decorate
 
-make_region = CacheRegion
+def make_region(*arg, **kw):
+    """Instantiate a new :class:`.CacheRegion`.
+    
+    Currently, :func:`.make_region` is a passthrough
+    to :class:`.CacheRegion`.  See that class for
+    constructor arguments.
+    
+    """
+    return CacheRegion(*arg, **kw)
 
-
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.