Issue #2181 resolved

Thg breaks symlink when mercurial.ini is symlinked on Windows 7

Oliver Schneider
created an issue

I'm using Thg version 2.5 and noticed that when I go and edit the mercurial.ini, which I symlinked into a different location, it breaks the link. Instead an ordinary file will appear there. This suggests that Thg is first deleting the file and then creating it anew.

Normally symlinks should be handled transparently, but deleting it here causes the issue.

I'll have a look at the code to see whether I can do anything about it.

Comments (8)

  1. Yuya Nishihara

    On Unix, symlink is handled pretty well. The actual file is replaced, with no symlink change.

    Thg is first deleting the file and then creating it anew.

    To be precise, it creates new file, and then replaces the existing by new one. It's common technique for safety.

    I guess Python (or Mercurial's atomictempfile) cannot recognize NTFS symlinks. It is very new feature of Windows 7.

  2. Oliver Schneider reporter

    Actually it's a feature since Windows Vista and has been implemented using the same technique as Volume Mount Points and Junction Points. The technical term from MS is Reparse Points.

    Vista was introduced in 2006 so it isn't terribly new, no.

    Can you explain one thing to me, though? What's the safety concern with this technique? I just would like to understand it in order to be able to devise a proper fix/workaround. Thanks in advance.

  3. Steve Borho

    the issue is one of write permissions and race conditions. writing a tempfile and then renaming that tempfile to the final target filename is generally considered an atomic operation from the filsystem's point of view, so readers of the file always see a consistent file (either the old file or the new file) and not some partially re-written file.

    It sounds like on Windows that Python's os.rename() destroys the symlink property

  4. Oliver Schneider reporter

    Let me verify that for you with a simple script.

    However, can't we work around this by pointing it to the target of the symlink instead of the symlink? I wrote a C++ class to handle all kinds of reparse points and put it into the public domain (CC0) and I'm ready to create a Python implementation of all functionality or a subset and also put it in the public domain.

    Do you reckon this would help anything?

    Is it acceptable to use ctypes for this purpose or do you require a Python module implemented in C/C++ instead?

    Thanks.

  5. Yuya Nishihara

    I found bug report for Python 2.7: http://bugs.python.org/issue9949

    to use ctypes for this purpose or do you require a Python module implemented in C/C++ instead

    IMO, ctypes is good choice. We can avoid extra build cost just to call several Win32APIs.

    FWIW, you could avoid symlink by setting HOME environment variable. Mercurial search user settings form %HOME%\mercurial.ini, %HOME%\.hgrc, %USERPROFILE%\mercurial.ini and %USERPROFILE%\.hgrc.

  6. Oliver Schneider reporter

    Hmm, interesting. That would indeed provide a nice workaround. However, I think a fix is in order. So I'll get to work with an implementation.

    Main question: should I only consider cases where the last part (i.e. the "file") is a symlink or would you prefer a solution where all path parts are resolved.

  7. Yuya Nishihara

    I suggest writing it as a Mercurial extension, so that you can start with minimal workaround code.

    In this case, all you need is to replace os.path.realpath() by the one supporting NTFS symlink. This is the example how to wrap realpath() function:

    import os.path
    from mercurial import extensions
    
    def _realpath(orig, path):
        print 'realpath(%r) called' % path
        return orig(path)
    
    def extsetup(ui):
        extensions.wrapfunction(os.path, 'realpath', _realpath)
    

    For details, please read http://mercurial.selenic.com/wiki/WritingExtensions .

    See also:

  8. Yuya Nishihara

    wconfig: disable atomic rename due to possible unlink/rename race on Windows

    Though I don't know if it was caused by our QFileSystemWatcher or a virus scanner, there are several reports saying util.rename() failed at util.unlink or os.rename on Windows. So using atomictempfile might not be always safer than normal file operation on Windows.

    https://bitbucket.org/tortoisehg/thg/issues?q=in+rename https://bitbucket.org/tortoisehg/thg/issue/2286/

    Recent TortoiseHg version disables the watcher while running Mercurial commands in its thread, but it isn't easy to disable the watcher while saving config file. Config file isn't large, so there would be little risk to read half- written data.

    This also fixes #2181, the NTFS symlink issue.

    → <<cset ef778af80e48>>

  9. Log in to comment