MpWriter crashes when added document count == batch size

Issue #245 resolved
Thomas Waldmann
created an issue

code: see attachment

traceback:
{{{
$ python mpwriter_bug.py
Process SubWriterTask-2:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/tw/w/whoosh/src/whoosh/filedb/multiproc.py", line 127, in run
runname, segment = finish_subsegment(writer, k)
File "/home/tw/w/whoosh/src/whoosh/filedb/multiproc.py", line 48, in finish_subsegment
runname = writer.pool.runs[0]
IndexError: list index out of range
Traceback (most recent call last):
File "mpwriter_bug.py", line 22, in <module>
w.commit() # BOOM!
File "/home/tw/w/whoosh/src/whoosh/filedb/multiproc.py", line 249, in commit
self._commit(mergetype, optimize, merge)
File "/home/tw/w/whoosh/src/whoosh/filedb/multiproc.py", line 274, in _commit
results.append(self.resultqueue.get(timeout=5))
File "/usr/lib/python2.7/multiprocessing/queues.py", line 130, in get
raise Empty
Queue.Empty
}}}

does not happen with the SerialMpWriter (and also not with the standard writer).

Comments (1)

  1. Log in to comment