- changed status to resolved
Make Postman's context processor lazy
In particular, it would be useful (and spare trips to the database) if the postman_unread_count
variable could be made a SimpleLazyObject
.
Motivation: we have different views that do not actually need the variable, and templates evaluated within other templates that cause duplicated trips to the database.
Comments (10)
-
repo owner -
reporter I noticed already some time ago but did not have the time to investigate this further, that on each page load on our website there are 8 requests done to the DB to fetch the number of unread messages (
SELECT COUNT(*) FROM "postman_message"
). We have a template snippet that includes thepostman_unread_count
variable several times, which is the reason the value is queried more than once.This started happening after the change of the context variable to resolve this issue
#111. Upon further investigation that I finally managed to do, the culprit is the wrapping of the value in alazy
call. Per the documentation in Django’s code: “Results are not memoized; the function is evaluated on every access”, which causes the excessive database queries. Changing fromlazy
to aSimpleLazyObject
fixed the issue: the value is evaluated only once and then remembered. -
reporter @Patrick Samson Are you planning to release a fix soon or should we implement a temporary workaround?
-
repo owner - changed status to open
re-open ; 2019-03-02 first fix was SimpleLazyObject ; 2020-07-10 conversion to lazy() was supposed to be neutral, but it seems not. Under investigation.
-
repo owner There are pros and cons to choose between a cached value or a refreshed value for each reference to a context variable.
For example, csrf_token is SimpeLazyObject(), but sql_queries is lazy() to reflect the exact current state.
postman_unread_count should be stable during the rendering of a view, but on the other hand, it is not expected, in a basic view, to reference the variable multiple times.
Could you please check if your “template snippet” can be optimized with a cache:
{% with uc=postman_unread_count %} ... reference {{ uc }} as many times you want {% endwith %}
-
reporter I tried the caching optimization, and the result is as follows:
Not sure why the count cannot be referenced several times. For example, our template code (simplified) looks a bit like this:
{% if not postman_unread_count %} <!-- show “empty inbox” UI element --> {% else %} {% blocktranslate count unread=postman_unread_count trimmed %} {{ count}} new message {% plural %} {{ count}} new messages {% endblocktranslate %} {% if postman_unread_count <= 99 %} <!-- show “half-full inbox” UI element --> {% else %} <!-- show “overflowing inbox” UI element --> {% endif %} {% endif %}
-
repo owner I still don’t see the usage of the
with
tag. But instead 3 evaluations of the variable in this template example. -
reporter The snippet above is an example. Earlier you said “it is not expected, in a basic view, to reference the variable multiple times” – thus I provided the above snippet as an example of legitimate code that needs the variable several times. I think it is not out of the ordinary to use it like that..?
On our actual website, I wrapped the code (which is similar to the above) in a
{% with %} ... {% endwith %}
, as you suggested; this did not reduce the number of queries. Each time I add 1 more reference (within the {% with %} block!) to the variable that received thepostman_unread_count
’s value, there is 1 more query to the database. This is what the picture with the 9 queries shows.I just added a bunch of
print
statements into Django code, and got the following output. Apparently, even the{% with %}
tag does not evaluate the lazy expression and keeps it lazy. As you can see, each reference to thepostman_unread_count_value
variable (assigned to in the{% with %}
tag) results in a new evaluation.WithNode __init__() ctx={'postman_unread_count_value': <django.template.base.FilterExpression object at 0x7fc3e446efd0>} FilterExpression resolve() <Variable: 'postman_unread_count'> Variable resolve() var_name=postman_unread_count in context <class 'function'> ==> <function inbox.<locals>.<lambda> at 0x7f2d1c4a0a70> WithNode render() values = {'postman_unread_count_value': "<class 'django.utils.functional.lazy.<locals>.__proxy__'> 6"} FilterExpression resolve() 'postman:inbox' FilterExpression resolve() <Variable: 'postman_unread_count_value'> Variable resolve() var_name=postman_unread_count_value in context <class 'django.utils.functional.lazy.<locals>.__proxy__'> ==> 6 FilterExpression resolve() <Variable: 'postman_unread_count_value'> Variable resolve() var_name=postman_unread_count_value in context <class 'django.utils.functional.lazy.<locals>.__proxy__'> ==> 6 FilterExpression resolve() <Variable: 'postman_unread_count_value'> Variable resolve() var_name=postman_unread_count_value in context <class 'django.utils.functional.lazy.<locals>.__proxy__'> ==> 6 /root/.venvs/mydev/lib/python3.7/site-packages/django/utils/translation/trans_real.py:250: DeprecationWarning: Plural value must be an integer, got __proxy__ tmsg = self._catalog.plural(msgid1, n) FilterExpression resolve() <Variable: 'postman_unread_count_value'> Variable resolve() var_name=postman_unread_count_value in context <class 'django.utils.functional.lazy.<locals>.__proxy__'> ==> 6 FilterExpression resolve() <Variable: 'postman_unread_count_value'> Variable resolve() var_name=postman_unread_count_value in context <class 'django.utils.functional.lazy.<locals>.__proxy__'> ==> 6
-
repo owner You’re right. I’m trying to find if there is a solution (may be not). Meanwhile, a workaround is to force the evaluation to a string with a noop filter, such as:
{% with uc=postman_unread_count|lower %}
-
repo owner - changed status to resolved
Rollback to a SimpleLazyObject implementation in the 4.3.1 release.
- Log in to comment
Fixed in repo.