# dogpile.cache-dict-config / docs / build / usage.rst

 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 Mike Bayer bf5be08 2012-01-18 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 Mike Bayer bf5be08 2012-01-18 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 Mike Bayer b4b1b54 2012-04-07 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 Mike Bayer b4b1b54 2012-04-07 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 Mike Bayer bf5be08 2012-01-18 zzzeek b91b2ce 2011-11-21 Mike Bayer 8a6571a 2012-04-07 zzzeek e87b051 2011-11-17 zzzeek b91b2ce 2011-11-21 zzzeek e87b051 2011-11-17 Mike Bayer bf5be08 2012-01-18 zzzeek e87b051 2011-11-17 Mike Bayer bf5be08 2012-01-18 zzzeek e87b051 2011-11-17   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 ============ Usage Guide ============ .. note:: dogpile.cache is **not released or completed** at this time. Development is currently in progress and the current code is not yet functional. Overview ======== At the time of this writing, popular key/value servers include Memcached _, Redis _, and Riak _. While these tools all have different usage focuses, they all have in common that the storage model is based on the retrieval of a value based on a key; as such, they are all potentially suitable for caching, particularly Memcached which is first and foremost designed for caching. With a caching system in mind, dogpile.cache provides an interface to a particular Python API targeted at that system. In this document, we'll illustrate Memcached usage using the pylibmc _ backend, which is a high performing Python library for Memcached. It can be compared to the python-memcached _ client, which is also an excellent product. Pylibmc is written against Memcached's native API so is markedly faster, though might be considered to have rougher edges. The API is actually a bit more verbose to allow for correct multithreaded usage. A dogpile.cache configuration consists of the following components: * A *region*, which is an instance of :class:.CacheRegion, and defines the configuration details for a particular cache backend. The :class:.CacheRegion can be considered the "front end" used by applications. * A *backend*, which is an instance of :class:.CacheBackend, describing how values are stored and retrieved from a backend. This interface specifies only :meth:~.CacheBackend.get, :meth:~.CacheBackend.set and :meth:~.CacheBackend.delete. The actual kind of :class:.CacheBackend in use for a particular :class:.CacheRegion is determined by the underlying Python API being used to talk to the cache, such as Pylibmc. The :class:.CacheBackend is instantiated behind the scenes and not directly accessed by applications under normal circumstances. * Value generation functions. These are user-defined functions that generate new values to be placed in the cache. While dogpile.cache offers the usual "set" approach of placing data into the cache, the usual mode of usage is to only instruct it to "get" a value, passing it a *creation function* which will be used to generate a new value if and only if one is needed. This "get-or-create" pattern is the entire key to the "Dogpile" system, which coordinates a single value creation operation among many concurrent get operations for a particular key, eliminating the issue of an expired value being redundantly re-generated by many workers simultaneously. Rudimentary Usage ================= dogpile.cache includes a Pylibmc backend. A basic configuration looks like:: from dogpile.cache import make_region region = make_region().configure( 'dogpile.cache.pylibmc', expiration_time = 3600, arguments = { 'url':["127.0.0.1"], } ) @region.cache_on_arguments() def load_user_info(user_id): return some_database.lookup_user_by_id(user_id) Above, we create a :class:.CacheRegion using the :func:.make_region function, then apply the backend configuration via the :meth:.CacheRegion.configure method, which returns the region. The name of the backend is the only argument required by :meth:.CacheRegion.configure itself, in this case dogpile.cache.pylibmc. However, in this specific case, the pylibmc backend also requires that the URL of the memcached server be passed within the arguments dictionary. The configuration is separated into two sections. Upon construction via :func:.make_region, the :class:.CacheRegion object is available, typically at module import time, for usage in decorating functions. Additional configuration details passed to :meth:.CacheRegion.configure are typically loaded from a configuration file and therefore not necessarily available until runtime, hence the two-step configurational process. Key arguments passed to :meth:.CacheRegion.configure include *expiration_time*, which is the expiration time passed to the Dogpile lock, and *arguments*, which are arguments used directly by the backend - in this case we are using arguments that are passed directly to the pylibmc module. Region Configuration ==================== The :func:.make_region function currently calls the :class:.CacheRegion constructor directly. .. autoclass:: dogpile.cache.region.CacheRegion :noindex: One you have a :class:.CacheRegion, the :meth:.CacheRegion.cache_on_arguments method can be used to decorate functions, but the cache itself can't be used until :meth:.CacheRegion.configure is called. The interface for that method is as follows: .. automethod:: dogpile.cache.region.CacheRegion.configure :noindex: The :class:.CacheRegion can also be configured from a dictionary, using the :meth:.CacheRegion.configure_from_config method: .. automethod:: dogpile.cache.region.CacheRegion.configure_from_config :noindex: Using a Region ============== The :class:.CacheRegion object is our front-end interface to a cache. It includes the following methods: .. automethod:: dogpile.cache.region.CacheRegion.get :noindex: .. automethod:: dogpile.cache.region.CacheRegion.get_or_create :noindex: .. automethod:: dogpile.cache.region.CacheRegion.set :noindex: .. automethod:: dogpile.cache.region.CacheRegion.delete :noindex: .. automethod:: dogpile.cache.region.CacheRegion.cache_on_arguments :noindex: .. _creating_backends: Creating Backends ================= Backends are located using the setuptools entrypoint system. To make life easier for writers of ad-hoc backends, a helper function is included which registers any backend in the same way as if it were part of the existing sys.path. For example, to create a backend called DictionaryBackend, we subclass :class:.CacheBackend:: from dogpile.cache import CacheBackend, NO_VALUE class DictionaryBackend(CacheBackend): def __init__(self, arguments): self.cache = {} def get(self, key): return self.cache.get(key, NO_VALUE) def set(self, key, value): self.cache[key] = value def delete(self, key): self.cache.pop(key) Then make sure the class is available underneath the entrypoint dogpile.cache. If we did this in a setup.py file, it would be in setup() as:: entry_points=""" [dogpile.cache] dictionary = mypackage.mybackend:DictionaryBackend """ Alternatively, if we want to register the plugin in the same process space without bothering to install anything, we can use register_backend:: from dogpile.cache import register_backend register_backend("dictionary", "mypackage.mybackend", "DictionaryBackend") Our new backend would be usable in a region like this:: from dogpile.cache import make_region region = make_region("dictionary") data = region.set("somekey", "somevalue") The values we receive for the backend here are instances of CachedValue. This is a tuple subclass of length two, of the form:: (payload, metadata) Where "payload" is the thing being cached, and "metadata" is information we store in the cache - a dictionary which currently has just the "creation time" and a "version identifier" as key/values. If the cache backend requires serialization, pickle or similar can be used on the tuple - the "metadata" portion will always be a small and easily serializable Python structure. 
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.