Close searchers in docs examples

Issue #75 resolved
ErikW created an issue

I'm using django-haystack to access query Whoosh. I'm frequently getting a traceback from Whoosh saying "[Errno 24] Too many open files". I did some Googleing and it looks like most other people are having similar problems during indexing, not querying.

Here is a copy of the traceback:
{{{

!python

Traceback:
File "/home/app/app_env/src/django/django/core/handlers/base.py" in get_response
100. response = callback(request, callback_args, callback_kwargs)
File "/home/app/app/wsgi/../data/views.py" in search_for_seed
63. for s in seeds:
File "/home/app/app_env/src/django-haystack/haystack/query.py" in _manual_iter
126. if not self._fill_cache(current_position, current_position + ITERATOR_LOAD_PER_QUERY):
File "/home/app/app_env/src/django-haystack/haystack/query.py" in _fill_cache
133. results = self.query.get_results()
File "/home/app/app_env/src/django-haystack/haystack/backends/init.py" in get_results
431. self.run()
File "/home/app/app_env/src/django-haystack/haystack/backends/init.py" in run
362. results = self.backend.search(final_query,
kwargs)
File "/home/app/app_env/src/django-haystack/haystack/backends/init.py" in wrapper
50. return func(obj, query_string,
args, kwargs)
File "/home/app/app_env/src/django-haystack/haystack/backends/whoosh_backend.py" in search
308. narrow_searcher = self.index.searcher()
File "/home/app/app_env/lib/python2.6/site-packages/Whoosh-1.3.3-py2.6.egg/whoosh/index.py" in searcher
303. return Searcher(self,
kwargs)
File "/home/app/app_env/lib/python2.6/site-packages/Whoosh-1.3.3-py2.6.egg/whoosh/searching.py" in init
49. self.ixreader = ix.reader()
File "/home/app/app_env/lib/python2.6/site-packages/Whoosh-1.3.3-py2.6.egg/whoosh/filedb/fileindex.py" in reader
291. for segment in info.segments]
File "/home/app/app_env/lib/python2.6/site-packages/Whoosh-1.3.3-py2.6.egg/whoosh/filedb/filereading.py" in init
37. tf = storage.open_file(segment.termsindex_filename)
File "/home/app/app_env/lib/python2.6/site-packages/Whoosh-1.3.3-py2.6.egg/whoosh/filedb/filestore.py" in open_file
58. f = StructFile(open(self._fpath(name), "rb"), name=name, args, *kwargs)
File "/home/app/app_env/lib/python2.6/site-packages/Whoosh-1.3.3-py2.6.egg/whoosh/filedb/structfile.py" in init
65. self.map = mmap.mmap(fd, self.size, access=mmap.ACCESS_READ)

Exception Type: error at /data/poster/search_for_seed/35613/
Exception Value: [Errno 24] Too many open files
}}}

Comments (10)

  1. Daniel Lindsley

    Haystack doesn't explicitly close any of the searchers it creates. They usually fall out of scope quickly. Is there a benefit to explicitly closing them?

  2. Matt Chaput repo owner

    Just that it it closes its files, so if the searcher is leaked it doesn't keep a bunch of files open.

  3. Matt Chaput repo owner

    Good, now I know who to dislike for being that guy. Just kidding!

    My only excuse is that it's tedious to wrap examples in the docs with try/finally or import with_statement. And I'm super-duper lazy. OK, that doesn't sound like a good excuse. I'll try to fix up the docs :)

  4. Matt Chaput repo owner

    docs/docstrings: use "with" statement when creating searchers, so they get closed. fixes #75.

    note: most of this diff is due to indenting caused by "with" usage.

    b25aea00994b

  5. Log in to comment