[mercurial] obsolescence markers seem to be not exchanged over HTTPS anymore

Issue #17123 resolved
Aurélien Campéas created an issue

Or rather, not all the time.

Regression observed with hg 4.3, 4.6 and 4.7.

Official response

Comments (35)

  1. Aurélien Campéas reporter

    After a bit of testing it appears only https operations are affected. Which is still unfortunate since from corporate networks ssh is often not an option.

  2. Sean Farley

    @Colin Wood Did you turn your hg http service on in production? If so, are you sure you have the evolve settings and, more importantly, the evolve extension installed?

  3. Matt Harbison

    I'm seeing this too on https push with hg 4.9 and evolve 8.4.1. I don't think it's a client side issue, because I was able to push obsolete markers to a repo early last May. I would have been using 4.5 or 4.6 at the time.

    It also has a weird effect on PRs. Under edit, the correct changesets and file content are shown. But updating the PR shows the wrong content on the main screen. An example is here:


    @Sean Farley Can you see the difference between the two views?

  4. Sean Farley

    @Colin Wood is the one that wrote a vcs service for handling https traffic. These issues happening at the same time as him turning it on are not a coincidence. Unfortunately, I'm not in the position of being able to do anything about it.

  5. Aurélien Campéas reporter

    Pierre-Yves' experimental instance is up and running and I might well be using that quite soon. At least I'll give it a try. How about your own attempts in a similar direction (tough not gitlab, I remember well your objections to the possibility of gitlab accepting this) ?

  6. Pierre Augier

    Dear @Colin Wood it would be really great if you can check if the evolve extension is installed in your hg http service?

    Not being able to use evolve with http is a real issue for Mercurial users.

  7. Sean Farley

    I talked with Colin last week and he has muted this thread. If you want to escalate it, you (all of you) will probably need to raise support tickets.

    I believe the problem is that his new service is not using my hg extension that sends config options via env variables but I have no idea since it's closed source.

  8. Aurélien Campéas reporter

    What is this support ticket thing ? Is there another ticketing system out there ?

  9. Matt Harbison

    For the benefit of anyone else who stumbles on this, the response I got from support was this:

    The Evolve "evolution support for mercurial" was a Beta feature, it was never released to our core product.

    Currently, due to the shift in the product prioritization, we had to turn off this feature.

    We are discussing internally the efforts it will take to re-enable it in the future and continue to build on it.

  10. James Reynolds

    This now seems to affect SSH too about 75% of the time.

    If they are going to remove the feature, some communication would be great so I can plan how to migrate off bitbucket.

  11. Gary Sackett staff

    Hi Everyone,

    Many apologies for a lack of updates to this issue.

    As @Matt Harbison mentioned, our support for this was rolled out as a BETA program.

    Due to the shift in the product prioritization, we had to turn off this feature off.

    We currently have no plans to turn it back on in the immediate future. We will update this public issue if the status changes in the future.

    To address the communication piece - no excuses, we did not really effectively communicate the rollout, what our future plans were, or provide a good warning that we were turning it off. We recognize that this is something we need to improve for future, similar changes and will be working to make sure we prevent future occurrences of these types of situations.

  12. James Reynolds

    Thanks for the update @Gary Sackett ,

    I'm the CTO of a middle market company who relies on your product to ship our product in a continuous way multiple times a day to a production environment.

    Unfortunately, the apology isn't really good enough and does nothing to solve my current problem, and probably anyone else who has relied on the feature.

    The responsible thing to do would be to turn the feature back on with a deprecation timeline (which is industry standard, even for beta features) so your current customers can plan out a reasonable migration plan to either a new Workflow, DVCS, or DVCS hosting provider. Right now, I have no options. However, I can say for certain using bitbucket in any capacity is not in the option pool, now or in the future.

    I am frankly shocked at not only the poor level of communication, but also the lack of caring about your customers - and for a company which historically has prided itself on customer-first policy. I've personally been a bitbucket customer, despite the warts, in some form since at least 2010 and have been using bitbucket as our main DVCS provider for the six years I've been at my current company. This is certainly not the level of customer support that I have come to expect from Atlassian products.

    The logical question any other mercurial user should ask, is mercurial itself deprecated? Will that also be killed off with no notice? Given this example here, who knows? Mercurial development is all about evolve and topics in the future. Not supporting it seems to me that bitbucket intends to also deprecate mercurial as well.

    But then, why bother using bitbucket when github and gitlab both have a superior product in practically every way. The only differentiator is / was mercurial support. And that means continued future support.

  13. Aurélien Campéas reporter

    Bitbucket is now dead for us for all intents and purposes. Will have to move to a Mercurial powered gitlab instance soon (for our collaborative things) and the official gitlab thing (using hg-git) for public visibility. I completely second what has been written by James Reynolds just above.

  14. Houzéfa Abbasbhay

    Thanks for clarifying why we were seeing "lost" obsolescence markers.

    Using evolve / topic is the best workflow when working with mercurial. Much better than named branches / bookmarks / whathaveyou.

    Very sad to see bitbucket still thinks of evolve as a beta feature. It just so happens we've had a Mercurial event in Paris on May 28th discussing how awesome workflows based on evolve / topic are. :P

  15. Christophe de Vienne

    @Gary Sackett The least would be to warn users about this! We are now in a position were some of our repositories are totally messed out because we have no local copy with up-to-date obs markers. This is a huge pain for us. Is there any way you re-enable the feature for a week or so, so we can get back on our feet?
    Note: between the two company accounts we use, we have hundreds of repos, and we do pay for this. Losing information because of this will leave a bitter taste.

  16. Christophe de Vienne

    @Gary Sackett Not to mention that our CI is totally blocked until we migrate everything in a hurry somewhere else. Which means days lost for delivering some projects to our clients. This situation is really unacceptable when we pay for hosting.

  17. James Reynolds

    @Christophe de Vienne if you pull numerous times, you can find a worker (eventually) that has it enabled.

    You might be able to get back to a consistent state doing that.

    doing this in your hgrc might help:

    evolution.obsdiscovery = no

    If you see the message:

    “(skipping discovery of obsolescence markers, will exchange everything)
    (controled by 'experimental.evolution.obsdiscovery' configuration)”

    Then you found a worker that still has evolve enabled. If not just keep trying. It took 8 or 9 attempts for me to get one as of this morning.

  18. Christophe de Vienne

    Thanks, but I can hardly do that for a hundred repo in a hurry. Will try nonetheless but with little hope.

  19. Gary Sackett staff

    Hi Everyone,
    Your feedback has been clearly heard, and we will be turning this feature back on for HTTPS by end of next week. We will keep this issue updated with the latest progress.

    As a workaround until that time, we have updated our SSH infrastructure so that this feature will work properly over SSH. You can temporarily switch over to the SSH protocol until we have this back up and running for HTTPS.

    To switch over to using SSH for your repositories, please see https://confluence.atlassian.com/bitbucket/ssh-keys-935365775.html.

    Thank you for your patience.

    Thank you,

  20. Sean Farley

    Gary's a great guy. I hate that management made him the messenger in this case. Miss ya, buddy.

  21. Matt Harbison

    Thanks for the info Gary. Just to clarify, is it the intention to get back to the status quo ante (a beta feature, hopefully to be improved on in the future), or is this a short term stopgap to let the paying customers get their data as someone referenced above?

  22. Gary Sackett staff

    Hi Everyone,

    This should be working as of today for HTTPS now, as we have evolve turned back on. Please let us know if you hit any roadblocks.


  23. Log in to comment