Commits

Matt Chaput committed c2f1f05

Fixed add_postings_to_pool() to use Cursor.flatten() instead of Node.flatten().
Node.flatten() caused recursion error on very long words.
Cursor.flatten() is iterative instead of recursive.
Fixes issue #366.

Comments (0)

Files changed (2)

src/whoosh/writing.py

         for fieldname, fieldobj in self.schema.items():
             if (fieldobj.separate_spelling()
                 and reader.has_word_graph(fieldname)):
-                for word in reader.word_graph(fieldname).flatten():
+
+                gr = reader._get_graph()
+                cursor = gr.cursor(fieldname)
+                for word in cursor.flatten():
                     # Adding a post where docnum=None marks it as a separate
                     # spelling word
                     add_post((fieldname, word, -1, -1, emptybytes))

tests/test_spelling.py

     assert c.format_string(highlight.UppercaseFormatter()) == "dworska"
 
 
+def test_very_long_words():
+    import sys
+    length = int(sys.getrecursionlimit() * 1.5)
+
+    strings1 = [u(chr(i) * length) for i in range(65, 70)]
+    strings2 = [u(chr(i) * length) for i in range(71, 75)]
+
+    ana = analysis.StemmingAnalyzer()
+    schema = fields.Schema(text=fields.TEXT(analyzer=ana, spelling=True))
+    ix = RamStorage().create_index(schema)
+    with ix.writer() as w:
+        for string in strings1:
+            w.add_document(text=string)
+
+    with ix.writer() as w:
+        for string in strings2:
+            w.add_document(text=string)
+        w.optimize = True
+
+
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.