Bitbucket API change - breaks [ gateway.php --> deploy.php -> syncChanges() ]

[AndrewS]

The current project's syncing feature appears to be broken with the new Webhook API changes.

The gateway.php file, which calls the deploy.php file to syncChanges() using the JSON payload is broken.

We have made a comment here ( https://confluence.atlassian.com/display/BITBUCKET/Event+Payloads?focusedCommentId=774045711&#comment-774045711 ) discussing the issue.

Given you're the owner of the project your thoughts on how to continue syncing files - using the new JSON payload would be very insightful.

[Carlos Pardo]

I've just installed the project and I am facing the same problem: no POST data is received by gateway.php and the sync process does nothing.

After some digging I set up a POST service (https://confluence.atlassian.com/display/BITBUCKET/POST+service+management) pointing to the gateway.php and looks like the sync process works.

Hope it helps.

[Alexandru Lixandru]

Carlos, actually the OP refers to a different issue here: the POST service is going to be shut down at some point by BitBucket. They have recently introduced a new way to get notifications about pushes (WebHooks), but it looks like this new way is lacking some of the capabilities of POST service. Anyway, the POST service is still here for now.

Thanks for your message! Alex

[Andžs Pilskalns]

Hi, I just spent couple hours on fixing this issue. Firstly, I hate that comment draft's are not stored here...

• You can easily handle as well as POST service as webhook in gateway.php with this php://input hack

if(isset($_POST['payload'])) {
    $payload = $_POST['payload'];
} else {
    //Check if input is valid JSON and return to such state
    $check = file_get_contents('php://input');
    $payload = (is_object(json_decode($check))) ? $check : null;
}

if(!empty($payload)) {
    // store commit data
    if (get_magic_quotes_gpc()) {
        file_put_contents( $location . $file, stripslashes($payload));
    } else {
        file_put_contents( $location . $file, $payload);
    }

    ...

Commit data now is being stored, what next?

• Apparently, there is difference in payload structure between POST service and webhook I did manage create some hotfixes, but not to the end of issue fix. The last issue today for me (and maybe my efforts to fix this) is following:

When debbuging into API 1 and 2, I try to find commits data as it should (file name that has been changed), but can't get to it. In any way, I can dig down to commit, but can't get comment data. For example, for that URL would be:

https://api.bitbucket.org/2.0/repositories/user/repo/commit/cbb5e41c54e1758f7946a7d2d444b134b33c0974/comments

But in response I get only and empty

{
values: [ ]
}

I tried also rest browser, but it gives me the same. For any case, use basic authentication.

Here is my modified deployChangeSet function, you can see changes I have made in top of it. If there were no above issue with empty comments.

You can use my work, include into your version and finish till the end this to make it work with webhooks and API 2.0!

[Andžs Pilskalns]

Update (on separate comment to stand out): About same time I made an issue for that empty API on Bitbucket issue tracker. You can follow progress here: https://bitbucket.org/site/master/issues/11679/comments-api-return-empty-list-bb-14057

[Alexandru Lixandru]

Thanks @Pilskalns. The API lacks other vital piece of information, anyway (the list of files changed), so until BitBucket offers a solution, there's not much we can do.

After analyzing all possible options, I have opened a ticket with them: https://bitbucket.org/site/master/issues/11688/
Waiting for their feedback.

Thanks,
Alex

[Andžs Pilskalns]

Hi, @alixandru, if API 2.0 were working as documented, that info would be included into commits request. I know, it's one more request to API which could be avoided, but at least, it's a way how it's supposed to dig down to data. From there it would be easy to extract filenames and then get raw data from project.

Anyway, when Bitbucket solves one or other way, I will be happy to contribute to new Bitbucket sync, as for now I have inspected all code and how it works against Bitbucket. Moreover, I have an idea for '.syncignore' file, that would be at root of project and would show which files to ignore on sync. For example 'readme.md, .gitignore' etc. as these files are not needed on production environment. Or maybe this info should be included in config.php, it's and open discussion :)

Have a great day!

[Todd]

Hi Everyone,

I'm (relatively) new to auto-deployment (and Git) so please bear with me.

Just so I'm clear, am I correct in thinking this auto-deploy script currently does not work with either the (old) POST Hook or the new Webhooks? I ask because I've spent 2 hours fiddling with both options with no success.

Previously I was successfully using this script http://jamescollings.co.uk/blog/bitbucket-deployment-script/ with BB, but it seems to have stopped working and I'm not sure if it's the new API or operator-error on my part.

I really need to get auto-deploy working again and this "bitbucket-sync" script looked very promising but I guess it's not quite ready. Can anyone recommend another auto-deploy script that does work with the new BB API?

Thanks

[Andžs Pilskalns]

Hi, EkoEcho, I am on my business trip, but as far I have followed this issue the main problem still exists - BB is having switch from API 1.0 ti 2.0 and to new and "promising" webhooks. If you really need auto-deploy you can have following write following workflow:

1. BB reiceives commit to monitored repro
2. BB sends webhook that it has a change into repro
3. You ask from BB full repro zip and then
4. You unzip only files you need or whole overwrite - just like initial sync, but with overwrite.

Off course, this doesn't save any bandwith, but BB is not solving clearly visible API issue either... So, why don't get things up and running?

Andžs

[Todd]

Thank you for your explanation, Andžs.

Perhaps until BB fixes this problem I might be better off using a service like DeployBot etc.

[Ludwig Mair]

Hi,

first, great work. i installed it like descriped. it works perfectly as expected.

Only i recognize that only the Repository gets copied to the folder .git on the webserver accout. the real files do not get deployed.

i appreciate any hint or help to accomplish it

cheers Ludwig

[Alexandru Lixandru]

The webhooks service is different from POST service and provides different data in the payload, which is insufficient for this script's way of operating (namely, this script needs the list of modified files). This makes this script unusable with webhooks.

As long as the POST service is still provided by Bitbucket, this script will continue to function properly (using POST service integration), but a port to the new webhooks service WILL NOT be implemented.