Issue #889 resolved

Can't commit file over network share

Miquel Burns
created an issue

For some reason, TortoiseHG seems to open some file that results in attempts to commit (and pushing as well) on a repo on a Windows Network share will result in the following message: {{{ The process cannot access the file because it is being used by another process }}} When I close THG and use the hg command line, it works just file. I do have to close THG however. And it seems to be with any repo that is accessed over a mapped drive.

version 2.1+18-40846ddb2223 with Mercurial-1.9+2-24ecdbe7c5c8, Python-2.6.6, PyQt-4.8.3, Qt-4.7.1

Comments (58)

  1. Steve Borho
    • changed status to open

    Committing over a Windows network share is not a recommended work-flow, but we do still expect this to work.

    We're not holding any files open that I'm aware of. I'm somewhat concerned this could be a side-effect of using QFileSystemWatcher() to monitor files for modifications.

    It would be a lot more useful if we knew which file it considered locked. Could you try the commit from within the log window, while adding a --debug argument?

  2. Miquel Burns reporter

    Sadly it seems --debug is not adding anything new.

    Here's all the lines I get (with and without --debug) in case there's a subtle clue in the format of the error:

    <<List of files being commited>>
    transaction abort!
    rollback completed
    abort: The process cannot access the file because it is being used by another process
    

    Worked fine with 2.0.5 BTW

  3. FrancoisJ

    I got the same problem (cannot commit on windows share) when usung tortoiseHG

    % hg add --repository H:\Work in Progress\Project1 H:\Work in Progress\Project1\Phase 2/New Text Document.txt
    [command completed successfully Wed Jul 06 07:57:56 2011]
    % hg commit --repository H:\Work in Progress\Project1 --verbose --user François JEAN <fjean@xxx.xxx> --message=sdf H:\Work in Progress\Project1\Phase 2/New Text Document.txt
    Phase 2/New Text Document.txt
    transaction abort!
    rollback completed
    abort: The process cannot access the file because it is being used by another process
    [command returned code 255 Wed Jul 06 07:58:03 2011]
    

    If I run the command line HG, everything works fine.

    Previous version of tortoiseHG were working well on the same windows share.

    PS: After I press the "commit" button in tortoise, the foder I try to commit blinks many times in Windows Explorer before showing me the error message.

  4. Miquel Burns reporter

    A bit more digging, found that it doesn't seem to matter what window is open (including the about dialog from the context menu) causes this issue.

    And yes, I tested from the command-line with the about dialog being the only THG thing running.

  5. Steve Borho

    Even the about dialog? That's interesting. It doesn't even open the repository.

    What does the output look like when you add --traceback to the commit command line?

  6. Miquel Burns reporter
    transaction abort!
    rollback completed
    Traceback (most recent call last):
      File "mercurial\dispatch.pyo", line 87, in _runcatch
      File "mercurial\dispatch.pyo", line 675, in _dispatch
      File "mercurial\dispatch.pyo", line 454, in runcommand
      File "mercurial\dispatch.pyo", line 729, in _runcommand
      File "mercurial\dispatch.pyo", line 683, in checkargs
      File "mercurial\dispatch.pyo", line 672, in <lambda>
      File "mercurial\util.pyo", line 385, in check
      File "mercurial\extensions.pyo", line 137, in wrap
      File "mercurial\util.pyo", line 385, in check
      File "hgext\mq.pyo", line 3218, in mqcommand
      File "mercurial\util.pyo", line 385, in check
      File "mercurial\commands.pyo", line 1092, in commit
      File "mercurial\cmdutil.pyo", line 1189, in commit
      File "mercurial\commands.pyo", line 1087, in commitfunc
      File "hgext\mq.pyo", line 3106, in commit
      File "mercurial\localrepo.pyo", line 1033, in commit
      File "mercurial\localrepo.pyo", line 1112, in commitctx
      File "mercurial\changelog.pyo", line 244, in add
      File "mercurial\revlog.pyo", line 971, in addrevision
      File "mercurial\changelog.pyo", line 97, in o
      File "mercurial\store.pyo", line 377, in __call__
      File "mercurial\scmutil.pyo", line 231, in __call__
      File "mercurial\windows.pyo", line 263, in rename
    WindowsError: [Error 32] The process cannot access the file because it is being used by another process
    abort: The process cannot access the file because it is being used by another process
    
  7. Steve Borho

    Interesting, so it is definitely the revlog file that is being "locked".

    Can you provide any more info on the type of network share? O/S version, service packs, etc?

    Also for the client O/S.

  8. Steve Borho

    the changelog is one of the files we monitor explicitly

    Please try this little bit of skulduggery

    Open a cmd.exe, then enter:
    % set THGDEBUG=1
    % cd path-to-repo
    % thg ci
    

    By setting THGDEBUG in the environment before launching thg.exe, it enables some debugging output from the repository polling logic.

    Is the network shared accessed via a drive mount, or a UNC path?

  9. Adrian Buehlmann

    Oh. Hmm. We've heard bad things about Symantec in the past already:

    http://mercurial.selenic.com/bts/issue2225

    Mercurial can deal pretty well with AV scanners that open files with windows API CreateFile having shared delete set.

    AV-scanners that lock files into memory or don't set the shared delete flag when accessing files are not supported.

    I can't tell in what camp Symantec Endpoint really is, since I don't have it here (nor would I want to have it here).

    Having mercurial repositories on Windows shares is not recommended anyway. Adding some obscure AV software on top and combine it with TortoiseHg's QFileSystemWatcher and your life might become interesting indeed.

  10. Miquel Burns reporter

    The debug thing doesn't seem to add anything.

    It's being accessed via a drive mount, but I believe I have the same issues with UNC paths (I did test for that)

  11. Nelson Almeida

    Hi everyone.

    I have the same problem when I do sync of repository or commit. I can sometimes overcome the problem using the update command, but sometimes doesn't work I have a drive mounted too, pointing to an server.

    I used version 0.8 and 1 of the tortoise and not have this problem.

    How we do when we can't work locally?

  12. Anonymous

    Don't know if it helps, but I get the same error using tortoiseHG. I also get it from the command line if I use:

    hg pull -u

    However, if I split this into two commands it works:

    hg pull hg update

  13. Steve Borho

    thgrepo: add option not to monitor changes on network drives (refs #889)

    Sometimes it causes locking issue to watch files on network share. This is just a workaround for the issue, but I think it's nice to disable monitoring on network drives.

    5fe1a71fa68b

  14. hg42

    current release 2.1.3 doesn't work for me! Does it include this fix? (edit: when looking at the commits list, it should) Bug behaviour is identical to older versions.

  15. matt wilkie
    • changed status to open

    I have this same error on a mapped network drive using TortoisHg Workbench (TortoiseHg 2.1.3 with Mercurial-1.9.2, Python-2.6.6, PyQt-4.8.3, Qt-4.7.1). Host system is Win7-Pro x64, server is Windows 2003 x64.

    As noted above, there is no problem using "hg --commit" from the command line on the same files and directory.

    I didn't have this problem when I last worked on this repository on 2011 June 28. Sorry I don't remember what version of THg I was using then; I upgraded 2 or 3 weeks ago.

    update: slight change in behaviour from previous bug reporters. `hg recover` on the command line after a thg abort shows the same file in use error. Running `hg recover` a second time immediately works. (I verified this cycle 3 times in a row.)

  16. matt wilkie

    My local working copy just got totally hosed!

    1. In workbench I tried to commit a change and got the abort message above about the process being in use.
    2. I flipped to a command shell and tried `hg commit`:
    rollback failed - please run hg recover
    abort: The process cannot access the file because it is being used by another process
    

    .3 I ran `hg recover` and all hell broke loose (edited for clarity):

    repository uses revlog format 0
    checking changesets
     changelog@?: index contains 30 extra bytes
     changelog@?: rev 1 points to unexpected changeset 0
     (expected 1)
     changelog@?: duplicate revision 1 (0)
     changelog@?: rev 2 points to unexpected changeset 0
     (expected 2)
     changelog@?: duplicate revision 2 (1)
    ...
     changelog@?: rev 642 points to unexpected changeset 0
     (expected 642)
     changelog@?: duplicate revision 642 (641)
    
    checking manifests
    warning: `manifest' uses revlog format 1
     manifest@?: rev 0 points to unexpected changeset 0
     manifest@?: fd3c32f87ac8 not in changesets
     manifest@?: rev 1 points to unexpected changeset 1
     manifest@?: d4f3dd672bd0 not in changesets
    ...
     manifest@?: rev 175 points to unexpected changeset 176
     manifest@?: ee3f8f33d00d not in changesets
    
    crosschecking files in changesets and manifests
     .hgtags@54: in manifest but not in changeset
     AlterAlias.py@0: in manifest but not in changeset
     Canvec Workflow.mm@28: in manifest but not in changeset
    ...
    
    checking files
    warning: `.hgtags' uses revlog format 1
     .hgtags@?: rev 0 points to unexpected changeset 54
     .hgtags@?: rev 1 points to unexpected changeset 57
     .hgtags@?: rev 2 points to unexpected changeset 138
    warning: `AlterAlias.py' uses revlog format 1
     AlterAlias.py@?: rev 0 points to unexpected changeset 0
     AlterAlias.py@?: rev 1 points to unexpected changeset 1
    ...
    warning: `show-updated-tiles.bat' uses revlog format 1
     show-updated-tiles.bat@?: rev 0 points to unexpected changeset 3
    
    775 files, 643 changesets, 997 total revisions
    1418 warnings encountered!
    3409 integrity errors encountered!
    (first damaged changeset appears to be 0)
    
    
    K:\code\Canvec\Scripts> hg recover
    no interrupted transaction available
    

    I archived the entire folder tree for forensic purposes if anyone is interested in doing that - http://files.environmentyukon.ca/matt/thg214_hosed_wc/code.7z, 7mb.

    All of my files appear to be intact, as in they still reflect the most recent edits, but all the interim commits since I last pushed to a remote repository are gone.

  17. Michael Jackson

    I was having this issue creating a new branch on a network share using this version: tortoisehg-2.1.3-hg-1.9.2-x64.msi

    I installed an older release, and it is now working fine: tortoisehg-2.0.5-hg-1.8.4-x64.msi

  18. Anonymous

    I just started having this problem, using 2.1.3-hg etc, so I just tried the newer 2.1.4 release (64 bit) and the repository in question still fails a pull.

    Guess I'll go download 2.0.5 ??

  19. Anonymous

    I found that using the option 'localonly' fixed the issue. TortoiseHG 2.2.1 and Mercurial 2.0.1. Win7 x64 client on Window Server 2003 SP2.

  20. Anonymous

    I need to resolve this, as I have high school students using THG on their network drives ( use of a personal network drive makes it possible for a student to use any workstation ) to push/pull from a central server. This bug seems to only be rearing its head on my Windows 7 machines, all of which are 64 bit.

    Is there a way to make localonly a global (as in global for all users on a computer) setting? The current docs don't seem to indicate a way. (I tried C:\Program Files\mercurial.ini before realizing that the documentation I was reading was for a previous version of THG.)

    I ask because I'm aiming for a situation where my students can work from any computer on campus without having to dig in and do a lot of configuring and fiddling on every machine they sit down at to make the program work.

  21. Matthew Farrell

    As Anonymous said, to work around this, in the tortoisehg settings for the repository set "Monitor Repo Changes" to "localonly"

    1. Right Click your repository in windows explorer
    2. Choose "TortoiseHg > Repository Settings"
    3. Change "Monitor Repo Changes" to "localonly"

    Make sure TortoiseHg is closed before you do this

  22. Anonymous

    Setting the Option to Local-Only has no effect if you use a mapped Drive. (e.g.: S: mapped to
    \server\xy).

    If i set the option to Local-Only and using a mapped drive i still get "The process cannot access the file because it is being used by another process", when working directly on the share "
    \server\xy" it works.

    The routine which checks for Network-Drives should be corrected to detect mapped drives also.

  23. Anonymous

    I can confirm. A network share mapped to a Windows drive letter has this issue for me on 64 bit Windows 7.

  24. Anonymous

    Experiencing exactly the same problem with tortoisehg-2.3.0-hg-2.1-x64 while using a mapped network drive.

    The older version of tortoisehg-2.0.5-hg-1.8.4-x64 worked well.

  25. Anonymous

    Same issue with THG 2.4.1 on Win7 x64, on a network drive that points to the local machine (e.g. `net use b:
    localhost\c$\users\%username%\code` I do this to avoid unecessarily\long\path\names\to\my\stuff)

    I upgraded to 2.4.2 and it seems to have gone away.

  26. Peter Doidge-Harrison

    This issue is also occurring for us. We have tried using a windows file share, and also using the Web Server on a dedicated host machine. We're on THG v2.7.1, Mercurial 2.5.2, Windows 7 clients. It occurs whenever we have two people pushing to the same repo. It appears that this issue has been open for over 18 months, and it's completely blocking our team workflow, so we're probably going to abandon mercurial at this point. :(

    Are there any updates pending?

  27. Gordon Palumbo

    We are seeing this way too often. THG 2.7.1 , Win 7 64bit. Z: is a Samba mount. Pull, commit, pull , no incoming changesets.

    % hg --repository Z:\Proj1 pull --verbose http://rhodecode:5000/Proj1 pulling from http://rhodecode:5000/Proj1 waiting for lock on repository Z:\Proj1 held by 'WINWKST:6460'

    The process specified is the workbench process itself. Sorry to ask stupid questions but 1. Can this be as simple as the workbench changing the working dir causing windows to be too aggressive? 2. Can the workbench monitor and if seeing the lock is itself, force the file descriptors closed, change working dir, etc and retrying automatically?

    This is impeding acceptance for a few people here.

  28. Yuya Nishihara

    waiting for lock on repository Z:\Proj1 held by 'WINWKST:6460'

    It says another Mercurial command or something is still running which locks the repository. Your problem seems different from this issue.

    Do you use any hook script or extension?

    1. Can this be as simple as the workbench changing the working dir causing windows to be too aggressive?

    Not sure what you mean, but probably no.

    1. Can the workbench monitor and if seeing the lock is itself, force the file descriptors closed, change working dir, etc and retrying automatically?

    As soon as the lock file is released, "pull" operation should continue.

    http://selenic.com/repo/hg/file/59cdd3a7e281/mercurial/localrepo.py#l984

  29. Yuya Nishihara

    repowatcher: do not monitor network/removable drives by default (closes #889)

    It tends to consume OS and network resources to hook events of remote files, and sometimes it causes strange problems.

    Changes made by TortoiseHg should be detected on commandFinished, so we shouldn't pay too much for filesystem monitoring by default.

    → <<cset cfb47d450cbb>>


    This change will be included in 2.11, i.e. Monitor Repo Changes = localonly by default.

  30. Coşkun ERGAN
    ** Mercurial version (3.1+2).  TortoiseHg version (3.1)
    ** Command: --nofork commit
    ** CWD: C:\Users\Coskun\Desktop\KULUCKA MAKINESI\kulucka-makinesi
    ** Encoding: cp1254
    ** Extensions loaded: 
    ** Python version: 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)]
    ** Windows version: sys.getwindowsversion(major=6, minor=1, build=7600, platform=2, service_pack='')
    ** Processor architecture: x86
    ** Qt-4.8.5 PyQt-4.10.3 QScintilla-2.7.2
    Traceback (most recent call last):
      File "tortoisehg\hgqt\status.pyo", line 586, in onCurrentChange
      File "tortoisehg\hgqt\filedata.pyo", line 214, in load
      File "tortoisehg\hgqt\filedata.pyo", line 387, in _readStatus
      File "mercurial\context.pyo", line 921, in data
      File "mercurial\filelog.pyo", line 38, in read
      File "mercurial\revlog.pyo", line 1025, in revision
      File "mercurial\revlog.pyo", line 1032, in _checkhash
      File "mercurial\revlog.pyo", line 1041, in checkhash
    RevlogError: integrity check failed on data/Output/KEIL/Project.hex.i:5
    

    What is the cause of the error?

  31. Log in to comment