git-mapfile is sometimes completely divorced from reality

Create issue
Issue #181 new
Cat's Eye Technologies created an issue

I maintain a large number of Mercurial repos on Bitbucket, with git mirrors on GitHub which are updated using hg-git. I'm using Mercurial 3.6.3 and hg-git 0.8.5 with dulwich 0.11.2.

For some of these repos (but not others,) I encounter the following error when trying to push to github:

pushing to git+ssh://
searching for changes
abort: branch 'refs/tags/rel_1_1_2014_0109' changed on the server, please pull and merge before pushing

...even though the tag has not changed on the server. I added some debugging to that part of the code, and saw that it thinks all the tags have changed on the server, which is certainly not the case. That prompted me to look at the git<->hg hash maps, and saw that, for the repositories where I do not get any error, they look fine, but for these problematic repositories, they look completely wrong.

Example where git-mapfile is right: (which is a mirror of )

$ cat guten-gutter/.hg/git-mapfile 
c593b4dd6cf8f28842c63a1d20c89f18b0ec58e3 34218fd29bb266064461cfb2b9d577a323d2585f
0aba23a052daa50525377027eeb9f4f015598a96 776a79c008756b473b252cceccc8cf2ca385bf5c
23deff9c39d1364d9b528d23850a78f0a0910db6 752e51a5e3341468b46bb0b20d3953a57af417ea
f537481631112baa618f1295f8f4f03f4bc679e9 5cc06d1a03d2b66917135aedd9c3f3f7c86daae1
da00148027dbb82536d03b0f37a73b37297dd653 38ce79d3627254d73701580ef977a682dc18d7f4
7932d77f25e0a8fac08135a0a59cd24c185367d5 4f163c49435102a6d05ef82ad2b467e31465f860
4ad5ef65f6a9d6310b51986d5b5de390c9b4becd 46f7a34c5bd231152f113ce87970c769cad3fb57
bde94807ef2f4bcb937de179c39f75e9fed897ff ad88d2cfbeec66646106af0f2a041421ce5fbce2
35c6f40f88cc9b117409e70224f14ecce81a2917 9e9fbea5f416590c9529e82f3c59dce7a4f7567c

You can look at any hash listed on and see that it exists in the above file.

Example where git-mapfile is completely wrong: (which is a mirror of )

$ cat jaccia/.hg/git-mapfile 
92e8ffd9107afc662a6322f0d9980575932d8d1a 936ce5f8f927919cfb4bf10c6cd7f1f2078687ab
3517e73466853256924c66b3578325d4948cb62c b549c5b881a6679b40cb5d5fbd56412f895fe406
a9f8328930a62b6e80fca07540a916e4f83a08e2 50392e0e0ef6c76244523d6885cb5d7c7588516c
210d2a2e8d6f5a9fbc28591048fe998dacf06b59 82d58bcbf3113250671223ae17238d5b8612fb5a
3f4f2f0d656d97a82059d94617e34e630e2f1aa7 90961d85ae265aa70d95de12a52f49069a72f5f8
f24726461b2a4e83aaeeff482a1abd8c5143513c 4df05fcd51d887b305172dc7c4a60e4fe3adae1f
41c4c060b7bc7bf480b52c6e24a872f9faf0fb68 989ac018fb25df78a3f9ac5dfc51313a7346b5ba
b77b6d958c447fc8c543b79924ccccfb082742c3 d3df0b128810d9a52a7ab64cccf6de1326288aee
594e4376a34ab3376a5f4733e7a4cb0188231d80 1adc54eb424874a87793693f89c38fa8697bcefb
efba61ef851201e0f9be24b205007fb354f067dd 1a942dbe1d872841c42e92b7e57bc7e74a3ba8d5
130b5cd17ab32820082015ecdf336fc51d83f73f d496f5f2f00ecbb5bdde042afa9689d9ffeef39d
1b4741aeb35a3c019a75b688df3d2ec9261aed68 25aeecf324b57be4a3b125f5b56fbbe6b6a777e5
7c7b6e494531b587c55f72ae27288e56d7f681bb 53f209387531f5227927f758dcae070fa0066a41
3cf4ab2a08f3818625492510bc08f957f0c6ee63 0d10ea65e716fbd1c2c966dc3902658da336822b
66dc66b83f52d05c37c61b50f63762c2086699e2 23328c9135b267738fa7b876acab6989b77cbdea
601de1453a45802357d880b3c66441bb272601ad 235f4614d2012afc9fa6459311550ea3f8365b05
891885f46cb26f4e0dc629d1e63e94cf2b245591 8521e549988e492a10fca8a53ced5f06c3431116
b1222806785b918ea82e2e1e79c41f8256156643 3a36c4a100187dce354146da2f7b8ce2cd62eb99
4560e81386575448282db11d61ba4320cc832768 b8c9ef2fc38663ed2a9a0152ac1bff22b186a2ef
cac855ab89350501c7ad78c4ee52b44d01af7bfa 126a48f127e048b21a789ef4f24f428485fa6699
63257fa3f4ea3f5f958aa2371ad63bf3a9a5ac94 707a320df6a2c74d7a950d0c40d49a5124f6d005
ea74167e37c13eda36538ba1030133fa91bc89df 1dea8634dd6206206b9cdb6c7430ee2caa058ab2

If I look at the hashes listed on ... I fail to find any of them in the above file.

If I delete the git-mapfile and regenerate it (by doing a failed push to github), it is no better - the same completely-wrong mapfile is produced.

Further, if I try to do a pull from github in this situation, it imports a second copy of all changesets into my hg repo, completely independent of my existing changesets. This is obviously wrong, and I assume it is a side-effect of this "hashes are completely different" problem - it thinks the changesets in the git repo have no relation to the changesets in hg.

Comments (2)

  1. Cat's Eye Technologies reporter

    Have found a workaround (of a sort.)

    The above configuration (hg 3.6.3, hg-git 0.8.5, dulwich 0.11.2) was reported from a machine running Ubuntu 14.04 LTS.

    I have two older laptops (call them laptop A and laptop B) which also have copies of all of these Mercurial repositories. Initially, they both experienced the same general problem when trying to push to Github -- the only difference being which of the repositories experienced the issue! (This suggests to me that it might even be an environment thing rather than a software version thing... hashing based on a different locale or something.)

    Then I fiddled with something on laptop B (and I'm afraid I don't remember what) and it appears to be able to mirror all of the repositories to Github... with the caveat that it takes about 10 minutes to push to Github. (I've seen this long wait in the past somewhere, and IIRC it went away when I upgraded dulwich. But that is not the case here.)

    The versions on these two laptops are:

    laptop A: Ubuntu 14.04 LTS, hg 3.4.2, hg-git 0.8.5, dulwich 0.9.9
    laptop B: Ubuntu 12.04 LTS, hg 2.0.2, hg-git 0.4.0, dulwich 0.11.1

    The 10-minute wait is annoying, but acceptable in my case, as these are not high-traffic repos nor high-availability mirrors.

  2. Александр Литягин

    i have problems, imho, similar to this on win xp working with github repository.

    problems starts after i have import one head into mq, change this head patch, and finishes it.

    striping changed head not helps. pull from source git repo - import nothig, push - aborts with errors

  3. Log in to comment