Absence of data not cached

Issue #56 new
created an issue

Perhaps this is by design or a known limitation, but I thought I'd file it anyway. Working on performance tuning Django CMS, I realized the following query is never cached when exists() is False:



PagePermission.objects.filter(can_view=True).exists() }}}

This is because this query returns None inside of has_results (django/db/models/sql/query.py line 424 in version 1.3.1.) It looks like johnny-cache doesn't distinguish between a cache hit of None and a cache miss:



val = self.cache_backend.get(key, None, db) }}}

Since Django seems to suggest not caching a literal None I'm not sure this is possible to fix. You could choose a different representation for None but it would need to be something that wouldn't collide with a valid string response. Maybe the key?

Comments (5)

  1. Mathieu Pillard

    FWIW, I'm getting strange issues in a project using johnny-cache ever since this commit (bcdb46c5d357)

    Deep inside django query engine, I'm getting "'NoneType' object is not iterable" errors. I digged a little, and found something interesting:

    # note: start with an empty cache
    from johnny.middleware import QueryCacheMiddleware
    q = QueryCacheMiddleware()
    len(Group.objects.filter(name__startswith="BUG")) # will return 0
    len(Group.objects.filter(name__startswith="BUG")) # will return 64, and django will even create dummy, non-existant, non-working Group objects for this query!

    (It fails with other objects, I used Group because it's pretty universal)

    64 being the length of the sentinel value, I don't think it's a coincidence :) I'm seeing this with both pylibmc and python-memcached backends, with django-1.3

  2. Mathieu Pillard

    Re-opening bug (priority should probably be changed as well), attaching a patch that solves the issue for me and add more tests. I've noticed SingleModelTest were using the standard django TestCase, and therefore happened inside a big transaction, skewing the results: only the local query cache is used, I think both the local and the "regular" query cache should be tested...

    With this new setup, another test is failing, but I don't understand why yet, so I'm posting the patch as-is hoping someone will figure it out...

  3. digitalemagine

    I confirm the bug!

    It's also easy to see it in the admin.

    First load of an admin page for an empty table will be empty. I refresh, and the admin page is full of all "None" objects. I have some filters, and they are one-character filters building up to the "sentinel" string

    I don't dare raising myself the priority, but this is actually quite severe in my opinion!

  4. Log in to comment