When a repo URL changes, GetComponents should offer more help

Create issue
Issue #367 open
Erik Schnetter created an issue

When a repo URL changes, GetComponents currently suggests to check out the repo from scratch (and presumably delete/ignore the old repo). This is very inflexible, in particular when the user has changes in the old repository, which often happens during development.

SVN (and presumably all other VC software) offer commands to update the URL, so that one can keep the current checkout. Of course, this makes only sense if this is the same repository that only moved to a new URL, but this seems to happen often enough.

GetComponents should at least output instructions for updating the URL.

It would be even better if GetComponents detected that new and old repo are the same, and then update the repo URL by itself.

Keyword:

Comments (12)

  1. Eric Seidel
    • changed status to open
    • changed milestone to ET_2011_05
    • removed comment

    I like the idea of updating the Repo URL automatically, but we'd need to be 100% certain that the new URL is in fact the same repo. With Git and Hg this should be fairly easy, since the commit hashes are cryptographically secure (I think we could just pick a commit at random and make sure it exists in both repos and the logs are the same). However, I'm not sure how we would do this in cvs or svn. Do you think the same idea would be secure enough?

  2. Erik Schnetter reporter
    • removed comment

    For svn we can compare UUIDs:

    Repository UUID: 17b73243-c579-4c4c-a9d2-2d5706c11dac

    We can probably just ignore CVS.

    Comparing a random commit is not good enough. It may be that two repositories are different branches, i.e. they would be 95% identical and then have diverged.

  3. Frank Löffler
    • removed comment

    Yes, for svn you can either lookup the UUID yourself, or you can try to avoid the internals and try a 'svn switch --relocate' and test for the exit value. This effectively also compared the UUIDs, but it does it for you and you are separated from any possible future change in the internals. Practically both give you pretty much the same, except maybe that even if you compare the UUIDs you would still need to do the relocation and check for its exit value to actually do the switch.

  4. Eric Seidel
    • removed comment

    I completely forgot about `svn switch`...

    Another possibility for git would be to add the new URL as a dummy remote, and run `git remote update <dummy>`. In the case that the repos are completely unrelated, git will emit a warning that there are no commits shared. However, it will finish the command without throwing an error, making this approach kinda clumsy. There's no guarantee that the warning will remain going forward (or that older versions of git have it). I'm also not sure if it would catch cases where the repos share a common ancestry, but have since diverged (my guess is that it would not).

    I'll have to look into the git/hg question more.

  5. Frank Löffler
    • removed comment

    Eric: do you think this could be included for the next release (quite soon)? If not, I would like to change the ticket to indicate that.

  6. Eric Seidel
    • removed comment

    I just pushed a fix for switching to new branches/urls in svn. I still have to look up strategies for git/hg though.

  7. Eric Seidel
    • removed comment

    For git, it appears that git actually does some internal verification when you try to fetch from a new remote.

    /Source/Cactus/repos/CRL$ git remote add test git@github.com:gridaphobe/dotvim /Source/Cactus/repos/CRL$ git fetch test fatal: did not find object for shallow 234bdec2c1fac30dc0bc8337df906ce752c88c73 fatal: The remote end hung up unexpectedly

    The problem is that the error message is not entirely consistent or informative...

    /Source/Cactus/repos/CRL$ git remote add test git:github.com/gridaphobe/dotvim.git /Cactus/repos/CRL$ git fetch test fatal: The remote end hung up unexpectedly

    Also, if I try via https instead, it just seems to hang.

  8. Frank Löffler
    • removed milestone
    • removed comment

    removing target milestone as this is solved for svn, and is very unlikely to be resolved for git and/or hg, as there isn't even an idea how to do this yet.

  9. Ian Hinder
    • removed comment

    Is it really solved for SVN? I have a Curie checkout and I manually did an svn switch on the manifest repository to get the new thornlist for the current trunk. I then fetched the CRL git repo and switched to the master branch to get the latest version. I then ran

    [ianhin@login-damiana Cactus]$ bin/GetComponents -a --update --root=. manifest/einsteintoolkit.th

    It first told me that the URL for simfactory2 had changed, so I moved it out of the way. It then told me

    Error: The URL for EinsteinAnalysis/Multipole has changed, please perform a clean checkout.

    Now, the URL will have changed because the branch has changed, but that is the only change, I think. Multipole is stored in SVN, so I don't think this is working, unless I've missed something.

    Steps to reproduce:

    1. Download Curie version wget --no-check-certificate https://github.com/gridaphobe/CRL/raw/ET_2011_05/GetComponents chmod 755 GetComponents ./GetComponents -a http://svn.einsteintoolkit.org/manifest/branches/ET_2011_05/einsteintoolkit.th

    cd Cactus

    1. Update GetComponents cd repos/CRL git fetch git checkout master cd ../..
    1. Update thornlist cd manifest svn switch http://svn.einsteintoolkit.org/manifest/trunk cd ..
    1. Move simfactory away (probably have to do this for Carpet as well) mv simfactory simfactory.old
    1. Attempt to upgrade to latest version bin/GetComponents --update --root=. manifest/einsteintoolkit.th
  10. Roland Haas

    git hash ccc00de "GetComponents: ignore git:// transport when creating canonical URL" of CRL helps a bit by making (for git only so far) GetComponents be more lenient when comparing URLs. Git also offers a “pushurl” setting that one can use to register both a “https:” to pull and a “git@” URL to push. Thus we could map URL to url and AUTH_URL to pushurl for git (which covers almost everything these days).

  11. Log in to comment