Commits

Matt Chaput committed ad25aae

Changed retokenizing code to include positions as well as chars,
since this might be useful for custom classes, e.g. scorers.

Comments (0)

Files changed (1)

src/whoosh/highlight.py

         else:
             # Retokenize the text
             analyzer = results.searcher.schema[fieldname].analyzer
-            tokens = analyzer(text, chars=True, mode="query",
+            tokens = analyzer(text, positions=True, chars=True, mode="query",
                               removestops=False)
             # Set Token.matched attribute for tokens that match a query term
             tokens = set_matched_filter(tokens, words)