TortoiseHg Workbench becomes increasingly slower as repository on network drive grows

Issue #3227 open
Sam Stenvall
created an issue

When a repository resides on a network drive (mainly a Samba share mounted in Windows) the whole GUI gets increasingly slower on most operations. A repository with < 100 commits feels just as if it were local, but on a repository with over 2000 revisions the whole thing slows down to a crawl.

The slowdown is most noticeable when selecting the current working directory (top row in the revision list) or when pressing the update icon in the changed files list.

I have tried all kinds of performance tweaks to the Samba server to no avail. The network share performs very good overall so I don't think the issue lies in the Samba configuration at all, rather in TortoiseHg itself.

This happens with all version of TortoiseHg (currently I'm using 2.7.2 but I can confirm that the issue still exists in 2.8.x).

If there is anything I can do to debug this issue, please let me know.

Comments (25)

  1. Angel Ezquerra

    I would strongly advise against the described workflow. Accessing remote mercurial servers through a windows share is risky because the locking mechanisms that mercurial uses to perform atomic writes to the repository do not work very well due to limitation on the samba protocol.

    Instead, I would recommend that you work on a local clone, and only access the server to do push and pull operations. Even then, if you have multiple users doing this you may have some issues, although those should be rare.

    I know this advice does not directly address your problem, but I really think that, if possible, you should change your workflow.

  2. Sam Stenvall reporter

    Yes, this seems to be the only "advice" people can come up with. I use this setup for web development (development is done on Windows while the repository is on a Linux virtual machine, made accessible to Windows via Samba). The workflow you described simply doesn't work in this scenario since I'd have to push/pull every time I save a file, which is (naturally) totally out of the question.

    I would hope someone could take a better look at this issue since there clearly is some scaling issues within the code, even though they only become apparent when accessing a slow filesystem (network drive in this case).

  3. Sam Stenvall reporter

    @Yuya Nishihara I already have that settings set to "localonly", I'm not sure if it helped very much.

    I have tried the other approach you suggested (sharing the folder from host to guest instead of vice versa) but it's difficult to use proper file permissions which is crucial in web development in order to spot potential issues in a real environment.

  4. Sam Stenvall reporter

    @Yuya Nishihara for reference the network share I'm using is mapped to a drive letter. When reading the bug report you linked I get the impression that it makes the repository seem "local" even when it isn't, so the setting doesn't really work in this particular case.

  5. Yuya Nishihara

    @Sam Stenvall TortoiseHg uses GetDriveType() API, so it should be recognized as a network drive.

    Random ideas:

    • Try hg stat or hg update from command-line. If it isn't fast, there's little room for improvement in our side.
    • If you use MQ, see #2329.
    • If you use branch filter, see #2559.
    • You can put source on Windows and store data in Linux, which will address 80% of permission issues.
    • If the VM has X11, you can use TortoiseHg on Linux (or X11 forwarding via SSH, etc.)
  6. Sam Stenvall reporter

    The foot icon only gets the globe (which I take it means it's a remote repository) if I use \servername\share\repo instead of Y:\repo.

    About your ideas:

    1. "hg status" from the command-line (on Windows where the repository is "mounted") takes 1,40 seconds while in TortoiseHg it takes about 3 seconds. On the Linux machine where the repository resides physically the same command takes maybe 0,1 seconds. Clicking the first row in the revision list takes about 3 seconds as well, even though the status bar doesn't say "Refreshing status".

    2. I have the MQ extension enabled but I rarely use it.

    3. I don't use the branch filter

    4. What do you mean by "source on Windows, store data in Linux"?

    5. I've tried that, instead of certain features being slow the whole program is slow (due to X11 being notoriously slow).

  7. Sam Stenvall reporter

    I'm still not sure if that option helps very much, the timings I posted in my last comment was taken when TortoiseHg detects the repository as remote (globe icon on the repository).

  8. Yuya Nishihara

    "hg status" ... 1,40 seconds while in TortoiseHg it takes about 3 seconds

    IIRC, TortoiseHg calls "hg status" twice, which is a bug. But even if it's fixed, 1.40sec is ~10 times slower than local access on virtual machine. It cannot be avoided.

    What do you mean by "source on Windows, store data in Linux"?

    Place source code on Windows and mount it from Linux, but store runtime data out of source tree.

    being slow the whole program is slow (due to X11 being notoriously slow).

    (If it isn't a SSH forwarding,) did you install kernel driver of that VM?

    The foot icon only gets the globe (which I take it means it's a remote repository) if I use \servername\share\repo instead of Y:\repo./local/

    The icon seems to be picked by another policy, probably by intention. f30747748bfd

  9. Sam Stenvall reporter

    Sounds like hg status being called twice is a known bug, if so why haven't it been fixed?

    Regarding the X11 forwarding, it was done over SSH (how else would it be done?) and all guest drivers were installed. Still, X11 is simply slow.

    The way the icon is selected is definitely confusing. Why would one want a mapped network drive to appear as local?

  10. Yuya Nishihara

    hg status being called twice is a known bug, if so why haven't it been fixed?

    Just because it needs more investigation, and there're many things to do, and we're not paid programmers for TortoiseHg.

    it was done over SSH (how else would it be done?)

    If VM is on the same machine, and if X11 server is installed in VM, you can just use it. X11 forwarding is slower than direct access.

    The way the icon is selected is definitely confusing

    Not sure, but it might be because of performance reason.

  11. Angel Ezquerra

    The most usual reason why a known bug is not fixed on any open source project, in my experience, is that nobody has been motivated enough to fix it :-)

    As Yuya said we are not paid to work on TortoiseHg, so we usually work on what we need or what we think is cool or what we think will help the most users. But our time is limited and we cannot do everything we'd like to do.

    That does not mean that we will never fix this problem. In fact I think it is an important issue that we should fix. I just means that we have been working on other things :-)

    Getting back to the icon issue, I don't recall this with a lot of detail but I think that checking if the path is a network drive was a bit costly, so we thought that checking that the path was a UNC path was a good compromise. My memory of this may be wrong though, so perhaps we should check again.

  12. Sam Stenvall reporter

    Don't get me wrong, I wasn't complaining.

    @Yuya Nishihara If I understand you correctly, that means I'd have to have the whole X server + GTK toolkit installed on the server and use it from the VM window, which is quite a huge workaround for a slow application.

  13. Angel Ezquerra

    Don't worry, I didn't think you were complaining, but I wanted to explain why sometimes a bug can take a while to get fixed.

    BTW, I will soon push a change that will let you change the 'Monitor Repo Changes' to 'never'. Hopefully this can help you, even if it is a little bit. A "shared drive" administrator could run a small script to set this setting on every shared mercurial repository on the drive...

  14. Angel Ezquerra

    settings: add 'never' option to 'Monitor Repo Changes' setting (refs #3227)

    One of the work arounds suggested for issue #3227 was to set the 'Monitor Repo Changes' to local only. However this may not work in some cases in which the UNC path has been mapped to a windows drive. In those cases it may be handy to completely disable the file system monitoring.

    → <<cset 51c04f2f5d9e>>

  15. Yuya Nishihara

    commit: avoid double refreshWctx when creating commit widget (refs #3227)

    If refreshwdstatus = always, refreshWctx() is called twice:
     0. createCommitWidget -> cw.reload() (queued request)
     1. tab changed -> _refreshCommitTabIfNeeded() -> refreshWctx()
     2. (delayed) cw.reload() -> refreshWctx()
    This patch eliminates refreshWctx() at 1.

    → <<cset 2aab40757c42>>

  16. Maxim S

    We have the same issue, slow hg repo on shared network drive. Enviroment is Win7 / WinXp + samba share on linux. A workaround found so far is to install "XP Mode" virtual machine (built-in Virtual PC effectively) extension on Win7 and run TortoiseHg from there. It speeds up the tortoise gui in 2 times approximately.

    Another note is that on some native XP machines tortoise gui works as fast as on virtualized boxes but on few it's as slow as on Win7 machines.

    Huge suspection is that large count of small file operations is slow over smb protocol implemented in Win7.

  17. Kaz Nishimura

    I have tried to see what are sent/received between a Windows client and a Samba server with Wireshark.

    I found Mercurial was trying to open some files with all uppercase name probably to check case sensitivity. It would take short time but could be noticeable if repeated in an operation.

  18. Log in to comment