Globally prefix all redis keys

Issue #70 new
Gustavo Carneiro
created an issue

Hi, I would like dogpile to prefix all keys it puts into redis with a string given by me. For instance, it now creates keys like


I want to prefix everything with a string, like "mymodule:dogpile:".

I found a way to do this, using the key_mangler, but it I'm not sure if it's the appropriate use for it. If it is appropriate, could you add some words in the documentation explaining this use case? Thanks.

Comments (6)

  1. Gustavo Carneiro reporter

    Well, it provides two examples of using this, neither of which is what I want. It would be nice to have a sentence like:

    Another scenario where this is useful is to add a prefix to all keys, globally, to avoid potentially collisions with other applications using the same memory storage.

    I don't like the way dogpile doesn't encourage prefixing. For a system like redis, the app most likely has other uses for redis, its own keys, and it always a good idea to consistently namespace everything, otherwise you risk getting key collisions. I managed to work around this using key_mangler, but it would be nicer if there was a setting like key_prefix in the redis backend itself, and examples updated to make use of it.

    Of course, I don't want to sound too negative so, I'll also add that I really really like dogpile.cache. It's so powerful while remaining easy to use! Nice work! :-)

  2. Michael Bayer repo owner

    well we have a "Recipes" section, send me a PR for something there. That Redis might often be used in a context where the same server is used for many different purposes is somewhat specific to Redis. This wouldn't be as common with memcached, file caching, memory caching, etc., though prefixes are still relevant there, which is why key_mangler is a region argument, not specific to a backend.

    dogpile's predecessor, Beaker, was built from the original "myghty" code, which enforced prefixes (called "namespaces") on everything, as itself came out of Perl's Cache::Cache. They were a totally obtrusive PITA that complicated everything.

    dogpile's job isn't to "encourage" anything, though. I don't really have time to educate people how to use caches. It's just a tool for people who already know what they want to get done what they need to get done and that's why key_mangler is there.

  3. Gustavo Carneiro reporter

    Well, I don't think a prefix is a PITA because it is a single prefix for the entire application that you would set once and then forget. OTOH, the namespaces in CacheRegion.cache_on_arguments can be more troublesome to use, because you have to repeat the namespace in all CacheRegion.cache_on_arguments calls everywhere. I think it makes more sense to have a namespace as a property of the CacheRegion itself.

    But, anyway, that's just some discussion. If you disagree with me, it's perfectly within your right.

    I'll probably send tomorrow some PR for documentation, if you don't mind.


  4. Michael Bayer repo owner

    right, we have the namespaces in the cache_on_arguments thing, where IMHO its a little more critical since we are generating keys.

    I'm fine with a per-region "namespace" but then we'd have to figure out how to make it sensible that it would take place not only for cache_on_arguments (easy), but for "get()" as well. that would be hard to explain.

  5. jvanasco

    I handle prefixes like this with a key_mangler on each region.

    IN a single application though, one redis database has over twenty different regions - each with different prefixes and expiration times.

  6. Log in to comment