fine-grained support for Cache Serialization and Cache Invalidation
jvanasco
Branch: jvanasco/dogpile.cache:feature-cachecontrol_region
Branch: zzzeek/dogpile.cache:master
Declined
Declined pull request
Rescinding this pull request.
Closed by: jvanasco·2015-11-11
I'm surprised I pulled this off so quickly, but tests are passing! This is fully documented too!
I expect this to be rejected as-is because the API implementation/details are not great -- but I'm hoping for feedback for resubmission.
This PR creates 2 new API objects that can be over-ridden:
CachedSerializer
Exposes a "loads" and "dumps" interface. Instead of callingpickle
/etc directly, backends register a new serializer instance. Regions can then configure backends to override the serializer. This way Redis will usepickle
by default, but one can switch to msgpack or even raw data if desired.CachedValueManager
. This has an awkward API and could use community feedback from. The general idea is that this divorces the 'cache wrapping' and timestamp expiration logic from dogpile.cache , and allows users to control this. By default this preserves the dogpile behavior viamanagers.TimestampCachedValue
, which keeps the CachedValue tuple and handles time expiry. All of the wrapping/serialization is handled via hooks, so "raw" data viamanagers.UntimedCachedValue
is supported.The goals in this approach: * Allow for "raw" data to be handled via the dogpile system, or msgpack encoding (ie, storing numbers/strings in redis) smaller size means more items can fit in the cache. without the use of pickle, data can be easily read by erlang/node/etc * The more "advanced" and aggressive users of dogpile constantly run into issues where they need to somehow alter the data, catch a hook on value generation, or expire based on custom business logic. By pushing this out of the
region
many of these advanced needs can be solved.Mainly addresses:
https://bitbucket.org/zzzeek/dogpile.cache/issues/18/get-serialization-to-be-configurable-with
But can be used to also address:
https://bitbucket.org/zzzeek/dogpile.cache/issues/45/update-expiration-time-on-get
https://bitbucket.org/zzzeek/dogpile.cache/issues/28/add-a-post-cache-load-hook
https://bitbucket.org/zzzeek/dogpile.cache/issues/37/expose-cache-age
And includes fixes for:
https://bitbucket.org/zzzeek/dogpile.cache/issues/87/redis-tests-flawed-because-backend-uses
https://bitbucket.org/zzzeek/dogpile.cache/issues/86/redis-tests-are-implemented-not-great
https://bitbucket.org/zzzeek/dogpile.cache/issues/85/testscache_fixtures_genericbackendfixture