1. Matt Chaput
  2. whoosh
Issue #9 wontfix

filelock: try harder not to leak fds (patch)

Stéphane Démurget
created an issue

(I hope creating new issues at bitbucket instead of the old Trac system is okay)

I've encountered http://whoosh.ca/ticket/70 (file lock fd not closed, that you fixed when importing whoosh into bitbucket -- this was the original point of my patch).

I'm opening a writer for each write in a FastCGI application (writes are not so frequent, but still, the system is exhausted of fds as the processes are reused).

Here's a small patch that tries to close the fd even if the flock fail. This also ensures the object state stays in sync (self.locked and self.fd). I'm silencing out eventual close errors not to mask the real exception raised until logging is available (I saw #6 here). I also fixed a missing self.locked update in the msvcrt case.

I do not have access to the box leaking fds on my side ATM but I should get a lsof or /proc/xxx/fd listing soon to ensure it's only the index not properly closed. Still, your fix might only fixes a part of #70 since the lsof output shows the fds of the opened segments (might only be the reporter's program indexing when lsof is launched, thus there is already no more leak).

Comments (5)

  1. Stéphane Démurget reporter

    Ok, I got the logs and only the lock fd is leaked indeed, so I just need to upgrade to 0.3.16. Anyway I think the above fixes are useful.

    thanks for Whoosh: it's API is really nicely written and fun to work with

  2. Thomas Waldmann

    clicking on the attachment "filelock-try-harder-not-to-leak-fds.diff" does just open the bug report page again.

    this bug is quite old and refers to very old whoosh version - is this bug still present?

  3. Anonymous

    hi, unfortunately, it's been 1 year I have ceased doing Python development, so I can not test again. I remember it was to ensure every fd was properly closed, as I had an issue with fd leak with a dev version.

  4. Log in to comment