Trigger webhook only for push to specific branch (BB-14642)

Issue #11728 open
Korijn Van Golen
created an issue

We trigger our Jenkins builds with a Bitbucket webhook, but it's firing for all pushes. We would like it only to trigger on pushes to a specific branch (e.g. release).

Similar to https://github.com/Nerdwin15/stash-jenkins-postreceive-webhook/issues/58

Official response

  • Daniel Tao staff

    Hey everyone:

    First, apologies for giving the appearance of not caring on this. That's certainly not the case. I think there is sentiment on the team that we'd like to add support for this. But let me back up slightly and provide some context to hopefully illuminate where things stand.

    The reason this issue was originally closed is that the issue description explicitly mentions Jenkins integration, and since there is a Jenkins plugin (maintained by the Jenkins team) that exhibits the desired behavior of only acting on pushes to specific branches, my thinking was that the need of the user who first created the issue was met.

    I recognize this does not imply that similar needs for all users were met; but it's worth pointing out that the reason the Jenkins plugin can do what it does is that our webhooks do include all the necessary branch information for integrations to provide filtering by branch. Generally speaking our philosophy with webhooks is that they should provide whatever information a consumer needs to act in whatever way is appropriate. Any functionality on top of that—for example custom payload transformations or filters—should be the responsibility of products integrating with webhooks. It's basically the same as with APIs: with any platform, the goal is to provide the tools for others to build whatever functionality they want.

    All that said, it's clear that enough users want this to be a built-in capability of Bitbucket (and presumably enough integrations don't support this desired functionality) that we can do a lot of good by implementing it on our side.

    So I do expect that we will work on this, but believe it or not it isn't as easy as it sounds. (When is it ever?) So it represents a relatively significant chunk of work that needs to spec'd and planned out.

    I can commit to updating this issue when we're further along in the process of delivering this.

Comments (144)

  1. Mengxin Zhu

    This enhancement would be really useful in below scenario,

    I have two jobs in CI, one for develop including building, testing and so on. The other is for production release, it will automatically deploy new build to productions after building source code.

    Currently the web service of jenkins will push commit events happening on both branch develop and product, it's will trigger unwanted build in Jenkins to make this event driven feature useless.

    Hope this help your team to understand the use case for this request.

  2. Daniel Tao staff

    The repo:push payload of our new webhooks includes all refs updated with each push. Build systems can use this information however they want.

    From what I can tell, the Jenkins plugin already handles this for you by only building when changes are pushed to branches you care about. You can specify these branches from the job configuration page, under "Branches to build":

    Screen Shot 2015-12-01 at 3.40.35 PM.png

    I've tested this locally using our new webhooks and the latest version of the Jenkins Bitbucket plugin, and it seems to work. @Korijn Van Golen am I missing something, or does this address your request?

  3. Korijn Van Golen reporter

    I guess the jenkins plugin is the catch here. I never noticed it's existece, I just connected to the repo's with hg or git directly. One would expect the branch filter to be listed with the other options on the bitbucket webhooks configuration panel.

    You should link to the plugin (and for other CI systems as well perhaps) on the webhooks configuration page, to drive people towards the solution.

  4. Tobias Hinz

    that jenkins plugin trick is neat but im integrating an ansible script for deployment at jenkins.

    ansible is taking care of the git checkout, its doing the composer install, cleanup etc. and the ansible script is coming from a totaly different repo than the actual repo i want to "watch".

    still: trigger webhook on push for a given branch is highly needed!

  5. Mikael Vallerie

    What if we're not using Jenkins, but something else (as Drone CI, or any other CI system) ? Should we wait for / develop one plugin for any build system ?

    To my mind this is still a major feature missing. This is doable with a 3 lines post-receive hook....

    No offense, but the "use the plugin" answer sounds like "we don't want to do it, guys".

  6. John Hunt

    Lol, this is totally not resolved and is a blatant case of 'we can't be bothered' OR more likely because it steps on Atlassian's bamboo.. don't expect this feature any time soon.

  7. ronn abueg

    Need this feature too but what also works for us is to simply execute the following command (as part of the build script in Jenkins): curl -s -d "payload={\"text\": \"YOUR_MESSAGE\"}" "WEBHOOK_URL"

  8. Zachery Woods

    I have already voted for this feature, but it really wasn't terribly complex to check the JSON data sent over and only rebuild based on that. However it would have saved a few hours of dev time and testing if it only pushed on specific branches.

  9. Daniel Tao staff

    Hey everyone:

    First, apologies for giving the appearance of not caring on this. That's certainly not the case. I think there is sentiment on the team that we'd like to add support for this. But let me back up slightly and provide some context to hopefully illuminate where things stand.

    The reason this issue was originally closed is that the issue description explicitly mentions Jenkins integration, and since there is a Jenkins plugin (maintained by the Jenkins team) that exhibits the desired behavior of only acting on pushes to specific branches, my thinking was that the need of the user who first created the issue was met.

    I recognize this does not imply that similar needs for all users were met; but it's worth pointing out that the reason the Jenkins plugin can do what it does is that our webhooks do include all the necessary branch information for integrations to provide filtering by branch. Generally speaking our philosophy with webhooks is that they should provide whatever information a consumer needs to act in whatever way is appropriate. Any functionality on top of that—for example custom payload transformations or filters—should be the responsibility of products integrating with webhooks. It's basically the same as with APIs: with any platform, the goal is to provide the tools for others to build whatever functionality they want.

    All that said, it's clear that enough users want this to be a built-in capability of Bitbucket (and presumably enough integrations don't support this desired functionality) that we can do a lot of good by implementing it on our side.

    So I do expect that we will work on this, but believe it or not it isn't as easy as it sounds. (When is it ever?) So it represents a relatively significant chunk of work that needs to spec'd and planned out.

    I can commit to updating this issue when we're further along in the process of delivering this.

  10. Jeyprakash Michaelraj

    For what its worth, specifying the branch on the Jenkins plugin is not robust (esp when you have slash in your branch names, and this is a common practice) It just does not work reliably. The fix should be on the trigger side. +1

  11. Avneesh Mishra

    I was so surprised and realised how branch specific merge/push request could be sent to Jenkins from Bitbucked and I found this via google!! So this feature is not supported by Bitbucket :(

  12. r

    @Daniel Tao The bb webhook to jenkins doesn't work if you have security turned on which means you have to use another plugin such as the Build Token Root Plugin. This seems to cause the branch specifier in the Jenkins plugin to be bypassed as it is building for each commit. This scenario isn't Bitbucket's issue but it gives another reason why this should be implemented on bitbucket's side.

  13. Sandeep Varshney

    Well @Daniel Tao +1 My team also need this.

    You are trying to give reason not to have this functionality in Bitbucket as there is Jenkins plugin available. But in the latest version of Jenkins 2.x, pipeline is not able to handle this case.

    So again we need this functionality from Bitbucket web side.

    Would be good if you can provide any ETA.

  14. Greg Peden

    Because this webhook event is fired on every update to any branch, if I update a development branch and then immediately push to live it causes two webook events to be triggered. Yes they each have different metadata about the originating branch, but nonetheless this causes Forge to perform two package updates in quick succession, even though in the moment the first event is fired there are no changes to the production branch, but on the second update right afterward there are. This causes Forge to generate an update error. As a result I have disabled the webhook feature and have been manually telling Forge to update. Probably Forge could fix it on their end, but my instinct is that it would be much much simpler to just add a branch name check in BitBucket to see if the event should be fired.

  15. Tobias Hinz

    i dont want a crappy workaround in php or any other language, im trying to set up here a professional integration pipeline. why is it so hard to implement this? should not be a big deal. but its a big show stopper. for me it just happend i had to stop my automated triggers because we tried to test something but... someone pushed something to another branch and bamm.... no crappy workarounds wanted. also no crappy non working plugins.

  16. Paulina Budzon

    How is this a "minor enhancement"? The Jenkins plugin doesn't check the payload, so triggers a build on change to every branch. No-one will be parsing the JSON, since the build should never be triggered in the first place! Not to mentioned people who don't use Jenkins!

    Stuff like that is the reason why I tell my clients to use Github instead of Bitbucket... And this ticket was created in 2015!

  17. t

    We are also looking for other solutions at the moment to build our deployment process / CI server. Is there any prediction will this be implemented anytime soon?

  18. Sandeep Varshney

    To All,

    who are looking to build PR from bitbucket. there is one alternative solution available.

    you can use Bitbucket pullrequest API to get the list of PR. but the limitation is it's showing only last 10-15 PRs only. it's ok something is better then nothing. so I am hitting that API on frequency level and getting the modify/created PR. I am pull the branch from that PR and compiling it.

  19. Mikael Vallerie

    Sort of solved this by switching from Jenkins to Drone CI >= 0.4, as you can filter the branch you wanna build, plus it has a some integration layer for many providers (aka Github, Bitbucket, etc) which are sufficient for our needs.

    Drone is smart enough in that case thanks to those integration layers. Still, that's far from something working on any CI server. To my mind, the only "global" way would be to include this feature in webhooks.

  20. Dominique Pijnenburg

    Would like to have the option to trigger webhooks for certain branches very much.

    I want to deploy a commit to the master branch automatically to websites and this is possible via webhooks. But of course I don't want this for the commits to the development-branch!

  21. Karel Juřička

    I found out that you can detect branch now in this payload:

    branch = push.changes[0].new.name
    

    In node.js is

     req.body.push.changes[0].new.name
    

    It's not ideal but it's conditional solution on deploy server.

  22. Dominique Pijnenburg

    By the way, my main purpose for webhooks is automatic pushing of a new version of a plugin or template for WordPress. For this I am using GitHub Updater and with that plugin it's possible to switch branches in WordPress itself. So that way it's possible to do what I want with webhooks.

    Still it'd be better to be able to configure webhooks related to branches.

  23. scharfmedia NA

    +1 !

    Damn it! I can't believe it after 2 years... Implement. This. Now. Finally! Its open for 2 years but instead of this useful ticket you waste your time on trivial design changes to the BB website! Instead of writing your official response from Daniel Tao you probably could have implemented this feature in same time. As paying customer i feel insulted by this kind of customer ignorance.

    Of course we still love you, just wanted to say it loud ;)

    EDIT: (i am responding below to Daniels response)

    Thanks that you read this Daniel. And i did not intend to insult you or the coder team. I think its pretty clear that the company on top (Atlassian) does make decisions which tickets your team shall implement and what not. Its not your developers fault and maybe not yours either. BUT i am critizing very much your company for not alotting time to this issue! So print my message or this channel list out, give it to your Head Head Chief of Whatever and say people are getting angry after 3 years of being ignored on that. Be thankful that we give you that feedback anyways.

    Everything which Atlassian is acquiring goes down the road, they have no feeling for what the community needs. We started using Hipchat... then it got acquired - it did not get the intention it needed and we switched to Slack. And now its happening for Bitbucket. And i can clearly tell you we will switch our next project to Github. Bitbucket was the better deal in the beginning, now its not anymore. And guess what, if Atlassians policy stays like this we will not use Jira or Confluence anymore in the future. The competition got really good, there is no need to stay on the Atlassian train anymore.

  24. Daniel Tao staff

    Wow, @scharfmedia NA. Tell us how you really feel!

    I'm not sure what sort of lightning coding skills you have, but I assure you that implementing this functionality (which involves changes to multiple independent services) will take us longer than it took me to write up my response. Generally I think 5 minutes communicating with our users is time well spent between the day's remaining hours, during which we are all busy delivering features on a regular basis.

    I should also add that Bitbucket's UX has been the subject of a significant amount of customer feedback, which we collect through surveys as well as direct customer interviews. So while our recent design changes may seem trivial to you, they were in fact a direct result of us listening to our users.

    At the end of the day we're a small team building a product that's used by millions of people. Since we can only do so much at a time, we have to prioritize; and sometimes that even means not being as responsive as we'd like on feature requests like this. That said, I will reiterate what I said earlier: we do want to do this, and I will be very happy to let everyone on this issue know when I have more concrete progress to share.

  25. Dominique Pijnenburg

    Thanks Daniel, you're totally right that it's a good thing to inform your users.

    I understand this is not the first priority after giving the UX such an overhaul. I also understand that it's not gonna take 5 minutes to create, test and rollout such a function. Would be nice if that was the case. We're just very anxious to see this improvement happen! :)

  26. Dominique Pijnenburg

    I'm using GitHub Updater for WordPress for a while now, but didn't yet figure out how to update against a certain branch, yet. Today I looked for help from the plugin editor and found out that adding '&tag=development' to the webhook will make it so that the webhook only works for commits on that branch.

    Maybe someone here can profit from this!

  27. Jon Sims

    +1. Slack doesnt give functionality to filter by branch unless you write an integration yourself. It seems that the bitbucket/slack integration code isnt open-sourced, so I can't just add it to the existing one :(

  28. Hamed Minaee

    Just to emphasis the need for that. aws does not have a built in functionality for bitbucket so I use a custom method to get the code using webhook trigger. But lack of webhook filtering by branch almost stuck everything in the project. Please do sth about it

  29. Bruno Watt

    Is this seriously still not a thing? This is making BB an impossible tool to use in the cloud (see AWS based statements above). I really dont want to use codeCommit but this means I really have no choice but to move ALL my customers off Atlassian products (you guys are getting worse and worse and worse).

  30. Andre Ouellet

    We are currently moving away from Bitbucket because of this.

    This issue/feature-lack means that even setting up this round about way of using Bitbucket in a AWS CodePipeline (see link below) is not possible, because it's un-usable when you can only get contents of master branch in the Bitbucket webhook.

    https://aws.amazon.com/blogs/apn/aws-sample-integrations-for-atlassian-bitbucket-pipelines/

    Does Atlassian realize that this issue means it's not viable for anyone to use Bitbucket with AWS Pipelines ? That's a market share not worth loosing IMO.

  31. nirpeled

    It's amazing that a company this big won't take the hour (probably less) needed and just add this feature already! I really don't understand why not just do it, instead - they are kinda saying "we don't give a shi%" about our users, not sure why :(

  32. Andre Ouellet

    @Daniel Tao

    Your last update on this ticket was 6 months ago.

    It's clear this will take alot of work to implement (it is never easy, I agree). It's clear that this is not a current top priority at Atlassian with all the other features and enhancements that Atlassian is currently working on.

    With the amount of feedback provided on this ticket in the last 2.5 years, and the frustration expressed, is there no way to at least get a discussion started within Atlassian about fixing this "missing" feature, which the other Git Code Repo vendors already have implemented for a while now ?

    Please let us know.

    Andre

  33. Jack Smith

    Here's a workaround we use for Jenkins for now, it can probably be adopted in other places too. Basic premise is that bitbucket sends a payload with their push hook, and one can filter on that payload to only react on specific branch:
    1. Install Generic Webhook Trigger Plugin
    2. In a new pipeline job configuration, under Build Triggers turn on Generic Webhook Trigger
    3. Add a parameter to that section named something like "branch", with expression "$.push.new.name" and JSONPath type.
    4. In the pipeline section of the job:

    if (params.branch == 'master') {
        build job: 'YOUR-REAL-JENKINS-JOB-NAME-GOES-HERE', parameters: [string(name: 'branch', value: params.branch)]
    } else {
        return 0
    }
    

    Supporting documentation:
    https://confluence.atlassian.com/bitbucket/event-payloads-740262817.html

    Hope this helps someone.

  34. Caroline Bartle

    Hi everyone! I wanted to let you know that Atlassian is close to launching a new Bitbucket Cloud for Slack app, and I'd like to invite those of you using Slack to join the beta group.

    The new integration features:

    • The ability to filter by branch so as to fine tune the volume of messages in your channels
    • More notification types than the existing integration
    • Interactive messages, so your team can perform Bitbucket actions without leaving Slack

    To get started with Bitbucket Cloud for Slack, simply click the link below to grant the app access to your Bitbucket team or user account.

    Install Bitbucket Cloud for Slack

    For more information you can check out the technical documentation.

  35. stefan

    Another Forge user here, automated webhooks to deploy to"test" and "production" separately obviously aren't possible through Bitbucket.

    Simple work around is to bookmark the trigger url in your browser and simply hit that url. A GET request will trigger the deployment. It's not fully auto but just requires a few clicks in browser.

  36. Manuel Feller

    Is there any ticket we can monitor, any timeline or any knowledge base how to only trigger a Post Webhook if there is a PR merged to one specific branch? There are lots of services out there that might need to get triggered after a merge to master, and I think it's a no go that each service has to provide a filter for something that should be done in the source repository... @Daniel Tao How can we help you that Atlassian finally puts focus on this feature? It is ridiculous that you need to craft regexes that exclude all possible branches but one just to make this possible...

  37. Sidney Bösch

    For now I created a bitbucket-pipelines.yml to make it work:

    pipelines:
      branches:
        staging:
          - step:
              script:
                - curl -X POST $WEBHOOK_STAGING
        master:
          - step:
              script:
                - curl -X POST $WEBHOOK_PRODUCTION
    

    Nonetheless it is really disappointing how Atlassian staff is handling this issue. Especially since Jenkins is not the only tool having issues with this.

  38. Adrian Ivanov

    Guys seriously? How hard is to implement this? It is 2019 now, that's 4 years. Even visual studio online has it too. Why are you staying behind? Out team has just migrated our repositories to bitbucket, and now we should modify all our deployment scripts to filter branches? That is nonsense.

  39. Log in to comment