fine-grained support for Cache Serialization and Cache Invalidation

Declined
#39 · Created  · Last updated

Declined pull request

Rescinding this pull request.

Closed by: ·2015-11-11

Description

  1. I'm surprised I pulled this off so quickly, but tests are passing! This is fully documented too!

  2. I expect this to be rejected as-is because the API implementation/details are not great -- but I'm hoping for feedback for resubmission.

This PR creates 2 new API objects that can be over-ridden:

  • CachedSerializer Exposes a "loads" and "dumps" interface. Instead of calling pickle/etc directly, backends register a new serializer instance. Regions can then configure backends to override the serializer. This way Redis will use pickle by default, but one can switch to msgpack or even raw data if desired.

  • CachedValueManager. This has an awkward API and could use community feedback from. The general idea is that this divorces the 'cache wrapping' and timestamp expiration logic from dogpile.cache , and allows users to control this. By default this preserves the dogpile behavior via managers.TimestampCachedValue, which keeps the CachedValue tuple and handles time expiry. All of the wrapping/serialization is handled via hooks, so "raw" data via managers.UntimedCachedValue is supported.

The goals in this approach: * Allow for "raw" data to be handled via the dogpile system, or msgpack encoding (ie, storing numbers/strings in redis) smaller size means more items can fit in the cache. without the use of pickle, data can be easily read by erlang/node/etc * The more "advanced" and aggressive users of dogpile constantly run into issues where they need to somehow alter the data, catch a hook on value generation, or expire based on custom business logic. By pushing this out of the region many of these advanced needs can be solved.

Mainly addresses:

But can be used to also address:

And includes fixes for:

0 attachments

0 comments

Loading commits...