1. Michael Bayer
  2. dogpile.cache
  3. Issues
Issue #28 new

Add a post-cache-load hook

Wichert Akkerman
created an issue

It would be practical if there was a way to detect that data was returned from the cache and then do some extra post-processing. My particular use case is caching SQLAlchemy ORM instances which needs to be merged into the session. Currently I am doing something like this:

def _expensive_function(foo, bar):

def expensive_function(foo, bar):
    result = _expensive_function(foo, bar)
    if result not in session:
        session.merge(result, load=False)
    return result

This could be reworked as a decorator to be more generic and easier to read, but it still has to do this in-session check and possible merge on every invocation instead of only for cache hits. It would be nice to be able to do this:

def _merge(obj):
    session.merge(obj, load=False)

def _expensive_function(foo, bar):

Comments (5)

  1. Michael Bayer repo owner

    I'm going to have this same issue soon as well, but for the most part your own decorator could just be on the outside:

    def my_function(...):

    we just lose the "invalidate" and "set_" functions that are placed on the decorator, is that the only issue ?

  2. Michael Bayer repo owner

    oh so you're saying, when the cache was not hit, the current Session is the one that's used to do the lookup? I see so here you're looking for a hook that only applies to cache-returned values. I'm thinking a service like that might want to be explicit about all three scenarios: wrap_cache_returned, wrap_creator_returned, wrap_returned.

  3. jvanasco

    i had a similar problem, and came up with a bastardized solution.

    Storing into the cache: I turn the objects column values into a dict , I ignore the relations.

    Pulling form the cache: I turn the stashed value into a dict that is overloaded to allow for attributes. I turn the id-based relation columns into lazyloaded functions to pull other objects from the cache

    It's annoying to configure a separate set of objects, but the "dict with attributes" and lazyloaded functions give me almost the same interface as the SqlAlchemy objects. None of my "read" operations (templates, views, logic) needed to be changed; all my "write" happened with fresh sqlalchemy objects off the db.

    I dropped a version of this framework on github https://github.com/jvanasco/pyramid_caching_api

  4. Log in to comment