Problem with large repositories.

Issue #122 resolved
Anonymous created an issue

It seems Murky constantly refreshes status on the repository. On very large repositories each refresh can take many seconds or minutes where the application pinwheels.

Events that seem to cause Murky to refresh: - When Murky gets focus. - When pushing the "Commit" button. - After a commit dialog says there are other files to be added and you push Cancel. - Some times when selecting files by clicking on them.

Needless to say it's annoying to have to keep focus on Murky while it's refreshing lets it lose focus and when you go back to Murky it just wants to refresh again.

It seems like a prudent way to deal with this is to not automatically rescan ever and just when the user selects it from the menu. Possibly add a rescan button next to Commit/Commit All.

Comments (7)

  1. Anonymous

    We have a repo with many thousands of files and many thousands of revisions. Murky bogs down a fair bit on this one.

    It could do what tortoisehg does — only grab the last few hundred changesets by default, and fetch more when the user asks.

  2. Jens Alfke repo owner
    • changed status to open

    Murky listens for filesystem activity in the repository (using the FSEvents API). It tries to be smart about coalescing, and scanning only the subdirectories that changed. The automatic refreshing is very convenient for repositories that aren't huge, and it's more Mac-like too (note that the Finder has always auto-refreshed.)

    There have been performance issues in the past with large repos, mostly involving stuff other than the direct hg operations, i.e. sorting or tableview updating. These were fixed, but there may well be more, or things may have regressed. The best thing you can do is to sample Murky while it's locked up refreshing, and attach the sample here.

    In the worst case we might have to add a heuristic to disable auto-refresh for large repos, but I'd rather see if the performance problems can be fixed.

    For the time being, you can disable auto-refresh by editing HgRepository and commenting out the call to "[_watcher start]" (currently on lines 88-89.) There is a manual Refresh menu command already.

  3. Anonymous

    Thanks for the workaround, I'll try it out.

    If you want an example of what I'm looking at, With SVN get the Eiffel source code repository

    svn co EiffelStudio

    The repository is 50,000 files so that's about 100,000 when SVN gets done with it. Then create a Mercurial repository over the checkout directory.

    Then edit some files in the repo and poke around, it takes around 30-45 seconds for status to do its job and it does it constantly.

  4. Anonymous

    The workaround was sound. The building instructions were clear and straight forward.

    I agree that if the auto-refresh functionality could be fine-tuned it would definitely fit in line with the OSX look-and-feel and I would use it.

  5. Anonymous

    Another use case for this issue is that it refreshes after selecting files to be added or removed from the repository. If I want to commit changes I have to 1) Refresh 2) Remove files 3) Automatic refresh 4) Add files 5) Automatic refresh 6) Commit all 7) Automatic refresh.

  6. Jens Alfke repo owner

    I checked out the Adium repository (3167 changesets right now) and used it to do some profiling. For some reason the "hg log" command, with my custom output style, was being incredibly slow, like 30 seconds with 100% CPU usage! Looks like Python was doing some massive string concatenation. But a normal 'hg log' is very fast, as is one using the new Hg 1.5 built-in XML style.

    I played around with it and found that for some reason the "{repo}" attribute in the style, which I wasn't even using, is what slowed it down. That's pretty weird since that attribute occurs only once in the header, not for every changeset. I took it out, and now the 30-second log command takes 0.7 seconds!

    In short, this should hopefully fix the main performance problem with large repositories.

  7. Log in to comment