1. Joao Pedro
  2. python-inotify
Issue #1 new

Incorrect paths after a directory is moved

Joao Pedro
repo owner created an issue

When moving a directory inside a watched path, any subsequent activity in the new path will be incorrectly reported as happening in the old directory.

This can be seen be the output in the attached script:

Adding watch on 'example'
Activity on 'example': [u'IN_OPEN', u'IN_ISDIR']
Activity on 'example': [u'IN_CLOSE_NOWRITE', u'IN_ISDIR']
Activity on u'example/dir0': [u'IN_OPEN', u'IN_ISDIR']
Activity on u'example/dir0': [u'IN_CLOSE_NOWRITE', u'IN_ISDIR']
Renaming 'example/dir0' to 'example/dir0_renamed'
Activity on u'example/dir0': [u'IN_MOVED_FROM', u'IN_ISDIR']
Activity on u'example/dir0_renamed': [u'IN_MOVED_TO', u'IN_ISDIR']
Activity on 'example/dir0': [u'IN_MOVE_SELF']
Creating directory in 'example/dir0_renamed/dir1'
Activity on u'example/dir0/dir1': [u'IN_CREATE', u'IN_ISDIR']

Comments (8)

  1. JanKanis

    Hi Joao,

    This is a hard problem. Your commits only solve the more general problem in a limited case. I've attached a modified move_test.py that demonstrates a different case which your changes do not fix. What could happen is that any of the parent directories of a watched directory are moved, and then the path of the watched directory and it's subdirectories are no longer correct. Or there could be symlinks/bind mounts/etc involved which complicate the issue even more.

    To really solve the problem the module would need to keep watches on all parents of a watched directory (and possibly also any symlinks involved), and keep a representation of the directory tree under a watched directory.

    I don't think I want to add these changes, because they add complexity and surprise without really solving the underlying problem that the path in a watch can be wrong, I guess (at the moment) one just has to accept that paths in watches can be wrong.

    I am working on a watcher that watches a single path (i.e. not all directories under that path) in such a modification-proof way by adding watches on every parent directory and symlink element, and automatically updating them if any of them change. It is almost done but not fully yet. Perhaps watching an entire directory tree could be built on top of that.

  2. Joao Pedro reporter

    Thank you for the feedback.

    Unfortunately, for the problem I am trying to solve right now, the current behavior of python-inotify does not make it a usable solution.

    I will keep using this version of python-inotify in the application I'm working on, until your new watcher is ready. For now, I think I will simply raise an exception whenever an IN_MOVE_SELF/IN_DELETE_SELF event for the watched path event occurs.

    Assuming that only regular files and directories exist inside the watched path, do you think that this approach will work for me?

    Nevertheless, I would like to to help in any way in order to solve this issue correctly. I can test your "newwatcher" branch or code parts of the solution. Just let me know :)

    Thank you.

  3. JanKanis

    Just out of curiosity, what is it you are using python-inotify for?

    I have pushed my development branch to bitbucket (I renamed the newwatcher branch to pathwatcher), it contains a partially working watcher that watches one single path including any symlinks and parent directories. A good tree watcher should probably have some of the same behavior as that has. I don't currently have a need myself for a watcher that watches a directory tree, but it certainly makes sense to have something like that added to the module.

  4. Joao Pedro reporter

    I'm testing it as a part of a file synchronization tool.

    Basically, I just want something I can depend on to watch a filesystem tree.

    I have been using the "watchdog" module [1] but I'm not completely happy with it. On several occasions events were missed. Until recently, moving a directory into the watched tree would not always work properly. The module was supposed to send events for all files and this was not always the case. I don't know if the situation has improved.

    python-inotify seems simple enough to understand, while having decent performance. It requires more work (e.g., scanning a subtree after receiving an IN_MOVED_TO event), but I'm fine with it.

    After reading Bryan O'Sullivan's opinion on "pyinotiy", I was a little wary to try it out (it is still an option, though).

    Your fork of python-inotify seems to be the most promising solution for the problem.

    I'll try out your pathwatcher branch and report anything I find.

    Thank you.

    [1] - https://github.com/gorakhargosh/watchdog

  5. Log in to comment