1. Matt Chaput
  2. whoosh
  3. Issues
Issue #378 resolved

Segment doesn't have a column for a sortable field

Chris Wilson
created an issue

Hi Matt,

I'm having some trouble with Whoosh 2.5.4 (through Haystack) where some of my Django models have an indexed published_date column, and others don't. I'm trying to sort by this column, in a query that's restricted to a model that does have it, and I get this error:

File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/haystack/backends/__init__.py", line 562, in run
    results = self.backend.search(final_query, **search_kwargs)
File "/home/chris/inaspauthoraid/django/website/intranet_search/whoosh_backend.py", line 331, in search
    **search_kwargs
File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/whoosh/searching.py", line 640, in search_page
    results = self.search(query, limit=pagenum * pagelen, **kwargs)
File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/whoosh/searching.py", line 787, in search
    self.search_with_collector(q, c)
File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/whoosh/searching.py", line 820, in search_with_collector
    collector.run()
File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/whoosh/collectors.py", line 144, in run
    self.set_subsearcher(subsearcher, offset)
File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/whoosh/collectors.py", line 619, in set_subsearcher
    self.child.set_subsearcher(subsearcher, offset)
File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/whoosh/collectors.py", line 566, in set_subsearcher
    self.categorizer.set_searcher(subsearcher, offset)
File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/whoosh/sorting.py", line 831, in set_searcher
    catter.set_searcher(segment_searcher, docoffset)
File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/whoosh/sorting.py", line 229, in set_searcher
    self._creader = r.column_reader(self._fieldname, translate=False)
File "/home/chris/inaspauthoraid/django/website/.ve/local/lib/python2.7/site-packages/whoosh/reading.py", line 895, in column_reader
    raise Exception("No column for field %r" % fieldname)
Exception: Failed to render template: page_3col.html: No column for field u'published_date'

Is this something you would expect? Is it to do with the way that Haystack uses a separate query to narrow the results down (a narrow_searcher) to the actual query (I guess in order to make sure the relevance information only depends on the real query?)

Is there any way around it? I'd just like to ignore any results that don't have a published_date, as they can't possibly be Resources.

Comments (7)

  1. Chris Wilson reporter

    Hi Matt, thanks for the quick reply! I've attached the index, please could you delete it once you've reproduced the problem, just in case it contains any confidential information?

    I don't know exactly what the query is, because it's made through Haystack. When I reproduce the bug in Django+Haystack and look at the stack, I can see this:

    results = self.search(query, limit=pagenum * pagelen, **kwargs)
    

    With local variables:

    pagelen == 10
    self == <whoosh.searching.Searcher object at 0xb225b70c>
    kwargs  == {'filter': <Top 1 Results for Term(u'django_ct', u'resources.baseresource') runtime=0.000514030456543>,
        'reverse': True,
        'sortedby': [u'published_date', u'django_id']}
    pagenum == 1
    query == NumericRange(u'published_date', None, 63521107200999999L, False, False, boost=1.0, constantscore=True)
    

    Does that help?

  2. Matt Chaput repo owner

    If a field has a column_type, but a particlular segment doesn't have a column file, the SegmentReader will now return an EmptyColumnReader with the column object's default value. This required changes to column-related APIs to change how reversed sorting keys are requested. Fixes issue #378.

    → <<cset c6af0c180946>>

  3. Log in to comment