Race conditions with temp storage

Issue #391 new
pombredanne NA created an issue

I think #368 is still not full resolved.
I am still getting the error with 2.5.7 where the fix is applied.

[Errno 2] No such file or directory: '/tmp/MAIN.tmp'

The thing is that I have many independent and unrelated processes creating temporary Ram-based indices.
/tmp/MAIN.tmp' is being deleted at times while other processes are using it or are trying to delete it.

Using tempfile and a real temp directory would be much safer IMHO

Comments (1)

  1. Martin Cech

    The Whoosh author Matt Chaput seems to resolve this in a way of merging the following PR: https://bitbucket.org/mchaput/whoosh/pull-requests/49/mkdir-of-temporary-index-directory-when/diff

    However an alternative and somewhat cleaner solution could be to use tempfile.mkdtemp() for creating random named temp directories which will then not collide when run in multi-process environment. Nate Coraor implemented it as such here https://github.com/galaxyproject/galaxy/pull/2310

  2. Log in to comment