Issue #8 new

Choices count not always accurate

evildmp
created an issue

Sometimes when date filters are applied, the counts for each choice are not correct.

  • it only seems to affect date choices
  • the wrong value affects the day, and then the month and year values reflect the error
  • when it happens, the count is usually larger than it should be by 1 or 2

Example: http://v029.medcn.uwcm.ac.uk:8000/new-news-archive/?date=2009&date=2009-04

Version: commit f867586

I can't work out where the error is creeping into the code. The list of items is correct; the error is in the {{ choice.count }}.

Comments (4)

  1. evildmp reporter

    This is what's happening:

    Here are the two queries we see:

    Django's:

    FROM `news_and_events_newsarticle` 
    LEFT OUTER JOIN `news_and_events_newsarticle_publish_to` ON (`news_and_events_newsarticle`.`id` = `news_and_events_newsarticle_publish_to`.`newsarticle_id`) 
    WHERE (`news_and_events_newsarticle`.`in_lists` = True AND `news_and_events_newsarticle`.`date` <= '2013-06-17 14:34:37' AND `news_and_events_newsarticle`.`published` = True AND (`news_and_events_newsarticle`.`hosted_by_id` = 2 OR `news_and_events_newsarticle_publish_to`.`entity_id` = 2 ) AND `news_and_events_newsarticle`.`date` >= '2009-04-01 00:00:00' AND `news_and_events_newsarticle`.`date` < '2009-05-01 00:00:00' AND `news_and_events_newsarticle`.`date` >= '2009-01-01 00:00:00' AND `news_and_events_newsarticle`.`date` < '2010-01-01 00:00:00' ) ORDER BY `news_and_events_newsarticle`.`date` DESC
    

    django-easyfilters's:

    SELECT CAST(DATE_FORMAT(`news_and_events_newsarticle`.`date`, '%%Y-%%m-01 00:00:00') AS DATETIME) as easyfilter_date_alias FROM `news_and_events_newsarticle` LEFT OUTER JOIN `news_and_events_newsarticle_publish_to` ON (`news_and_events_newsarticle`.`id` = `news_and_events_newsarticle_publish_to`.`newsarticle_id`) WHERE (`news_and_events_newsarticle`.`in_lists` = True  AND `news_and_events_newsarticle`.`date` <= '2013-06-17 14:10:54'  AND `news_and_events_newsarticle`.`published` = True  AND (`news_and_events_newsarticle`.`hosted_by_id` = 2  OR `news_and_events_newsarticle_publish_to`.`entity_id` = 2 ) AND `news_and_events_newsarticle`.`date` >= '2009-04-01 00:00:00'  AND `news_and_events_newsarticle`.`date` < '2009-05-01 00:00:00' ) ORDER BY 1 ASC
    ) subquery GROUP BY (easyfilter_date_alias) ORDER BY (easyfilter_date_alias)
    
  2. evildmp reporter

    I think this may help describe the issue:

    Until DateTimeFilter.get_choices_add_recursive().get_choices_add_recursive(), we're looking at querysets of items.

    However a DateTimeFilter needs to work with the sets of dates that these items falls into, so it obtains a queryset of dates, using date_aggregation().

    The problem is that it then continues to use this queryset of dates, rather than the queryset of items to provide the items count. To do this, in date_aggregation() it clones the query attribute of the queryset of dates, date_qs:

    date_q = date_qs.query.clone()
    

    This works, if:

    • date_q.distinct = False (which it does), and
    • the queryset returned using this cloned and modified query doesn't need distinct() to remove unwanted duplicate dates.

    This only works for querysets where we can guarantee that the number of items in the item queryset matches the number of dates in the dates queryset, when distinct = False is applied to the latter.

    If the queryset that we first pass to the FilterSet requires distinct(), then this is never going to work. We need to be counting items directly, not relying on their dates.

  3. Log in to comment