Add specific pipelines configuration for Pull Request events

Issue #13438 open
Marian Jureczko
created an issue

Pipelines are triggered by push. There is the possibility to configure a number of other webhooks, e.g. 'on pull request'. However it is not possible to connect them with a pipeline and disable or reconfigure the default 'Pipelines' webhook which triggers on push.

If it were possible to have different webhooks as triggers and distinguish between them inside the pipeline it would enable:

  • to configure different pipelines for different events (webhooks), e.g.:

    • just unit tests with fast feedback loop for every change on feature branch,

    • heavy, time consuming quality assurance for pull request;

  • better integration with "Sonar for Bitbucket Cloud", especially with respect to the "Shows Sonar code issues in pull requests" feature

Official response

  • Aneita Yang staff

    Hi everyone,

    No progress on this so far - the team are currently finishing up on some higher priority projects. However, this is on our medium term roadmap and we hope to have something to customers by the end of 2018.

    Thanks for your patience and thanks to everyone who's participated in a customer interview or provided feedback on this ticket.

    Aneita

Comments (83)

  1. Ed Mitchell

    Hello,

    Just having the ability to modify the trigger in the webhook would be enough. I really don't want to run a build on every single origin repo push. That's a easy way to run out of build minutes.

  2. P

    I am in the same boat. We run tests locally for each branch, but only want to run our full tests when a PR is created.

  3. Jeroen De Raedt staff

    Hi everyone,

    we're trying to understand the use cases for this feature request a bit more, and in the light of that we had a question around the configurability of this.

    More specifically, we would want to know whether there is any need to support different pipelines for different types of pull requests?

    pipelines: 
      pull-requests:
        master:
          - step:
              script:
                - echo "Executed when a pull request is opened against master branch"
        production:
          - step:
              script:
                - echo "Executed when a pull request is opened against production branch"
    

    The above example would execute the pipeline 'master' when a pull request is created to merge a branch to the master branch. This approach is similar to how you can define different pipelines for different branches. The difference here though is that for pull requests we would use the destination branch to identify the pipeline (and not the source branch to which you pushed a change). Alternatively we would only support 1 pipeline definition that would be applied to all pull requests:

    pipelines: 
      pull-requests:
        - step:
            script:
              - echo "Executed when a pull request is opened against master branch"
    

    Would it be useful to have the option to configure multiple pipelines based on the destination branch of pull requests, or would a single pipeline suffice?

    Thanks, Jeroen De Raedt, Pipelines Developer

  4. Ben Gummer staff

    @Jeroen De Raedt in AtlasKit we would only need a single config for PRs because we only develop feature branches off master. But having said that I can imagine that teams who "chain" branches together in a waterfall fashion would benefit from your 1st example config. (e.g. AUI used to have "5.8" -> "5.9" -> "5.10" branches, where changes made in "5.8" were merged all the way up)

    an important thing for AtlasKit would be ensuring that a new build is started when a PR is raised even if a non-PR build for the same source branch is already in progress

  5. Joël Wijngaarde

    Hi @Jeroen De Raedt, happy to hear you guys are looking into this request.

    The use case for us would be to make better use of the insights from SonarQube (using the plugin for BitBucket).

    We would like SonarQube to perform a code analysis on the proposed changes in the pull request. For SonarQube to be able to comment on a pull request the pipeline needs to run on the feature branch. If the correct pipeline can be run on the creation of a pull request it allows SonarQube to comment on the code in the pull request.

    For example we have a feature-branch from master. When people are working on this branch and push the code to bitbucket a pipeline can kick in to do various tests as is currently the case.

    When the developer decides it is time to merge his code into the develop branch he creates a pull request. Currently no pipeline is being triggered which would allow automated comments on the pull request (SonarQube or any other smart tool/ bot). If we could say that the same pipeline would also be triggered on a pull-request it would solve this use case.

    However I can understand that you want to define a custom pipeline in this case (one that only performs the automated commenting for instance). I am not sure if it is necessary to define a pipeline based on the target branch.

  6. Ed Mitchell

    @Jeroen De Raedt

    Thanks for taking a look at this ticket. I could see a use case for having a different scripts be run between different branches. I would prefer this behavior because all I really want is to tell bitbucket to run a pipeline on a PR creation and update, and the script that should be run should be specified by the branch designation that already exists in the YML file.

    I'm with Joël though, I could see a use for using pipelines on our release branch to run certain scripts and such.

  7. Marian Jureczko reporter

    I believe that the SonarQube is a great example. One usually execute a preview analyse on a feature branch. The preview analyse is executed against a baseline that should reflect the state of the PR destination branch. If there is more than one possibility, then it is critical to know which one is the destination branch. Having dedicated pipelines, as @Jeroen De Raedt suggested, addresses the issue well. Nonetheless, it would be more comfortable to have the branch name in a parameter and create a parametrized pipeline.

    The PR updates support seems to be a very important feature too. Consider following scenario. A developer creates PR which triggers pipeline. Pipeline output shows that the changes could be better, thus the developer pushes some improvements. It would be nice to rerun the same pipeline and have the output in the context of the same PR.

  8. David Wood

    @Jeroen De Raedt

    I have just run into the need today to be able to have certain checks run on pull requests, but only when they are destined for specific branches. Your first suggestion for format looks like it would be very flexible and accommodate most use cases, though I would also suggest allowing a method for running a pipeline on all pull requests regardless of branch.

    Our specific use case is to have automated code format checks as well as PHP Code Sniffer checks be run so we ensure a consistent format as well as ensuring common PHP mistakes do not make it into a pull request. Then once a pull request pipeline task passes it can be reviewed and merged in.

  9. Kevin Basarab

    @Jeroen De Raedt Your example code above makes a lot of sense. I don't foresee for our usecase needing to run different tests based on the target branch but I could see that being very useful for others. I love the idea of basing on PR or on push and being able to define differing tests for each.

    As others have mentioned the biggest user story I can think of is:

    On pushes I want a quick sanity check such as syntax errors or linting to quickly get feedback on my code and resolve.

    On pull requests I want to ensure my full suite of tests is run to ensure proper integration to the target branch

  10. AntonK

    we also need this,

    we're trying to run pronto

    pronto run -f bitbucket_pr -c origin/develop
    

    but only after PR create/update to see comments on pr board

    and in case of simple branch creation this will fault

  11. David da Silva

    Hi, @Jeroen De Raedt your example code is pretty much exactly what we would need.

    Being able to create pull-request triggered builds, targetting specific branches, would be ideal. (Would also be nice to block the merge until the build comes back, but can't look into that until this feature lands).

    Is there an ETA on this feature?

  12. Ian Colwell

    Agree! For our team, it is a complete waste of build minutes to build every commit. We only need to test the PR into master to ensure master stays solid. Travis CI does this very well, you can turn on/off "build on push" and "build on PR". I hope Bitbucket Pipelines can allow this customization soon.

  13. Misha Wakerman

    We also wanted this feature but needed a current solution to save build minutes.

    Have been meaning to write this up fully but we were able to achieve this using the existing pipelines API. Our pipelines still run for every push but end (almost) immediately without running tests unless there is a PR for the branch. The summary of the steps are:

    1. Use a base image with jq installed. Actually using a base image with all your build dependencies is good for reducing build time.
    2. Create an 'App password' that has 'Pull Request > Read' permissions.
    3. In the script block of our bitbucket-pipelines.yaml check if the branch has an open PR and only run tests if there is:
    script:
       - INDEX=$(curl --user <$USERNAME>:<$APP_PASSWORD> -s -XGET "https://api.bitbucket.org/2.0/ repositories/<$TEAM>/<$REPO>/pullrequests?pagelen=50&state=OPEN" | jq ".values[] | .source.branch.name  | index(\"$BITBUCKET_BRANCH\")")
       - IS_PR=$(if [ -z "$INDEX" ] || [ "$INDEX" == "null" ]; then echo false; else echo true; fi)
       - if [ $IS_PR != 'true' ]; then echo Skipping tests as branch not a pull request; fi
       - if [ $IS_PR == 'true' ]; then yarn test; fi
    

    Running tests on post-merge result

    We also wanted our testing to happen on the post-merge result of the pull request. To achieve this you need to take a full depth clone of the repository and do the merge as a part of our script. Our git workflow is that we only make pull-requests into develop but you could adapt this to handle other git branching styles. Our redacted bitbucket-pipelines.yaml file:

    # custom image hosted on docker hub with all build dependencies including jq
    image: assembletech/pipeline:latest
    clone:
      # required to do merge from develop into PR branch
      depth: full                         
    pipelines:
      default:
        - step:
            # caching not required but advised for faster builds
            caches:
              - node-server
            script:
              # check if there is a PR open for the current branch (note that pagelen=50 appears to be the maximum)
              - INDEX=$(curl --user <$USERNAME>:<$APP_PASSWORD> -s -XGET "https://api.bitbucket.org/2.0/repositories/<$TEAM>/<$REPO>/pullrequests?pagelen=50&state=OPEN" | jq ".values[] | .source.branch.name | index(\"$BITBUCKET_BRANCH\")")
              - IS_PR=$(if [ -z "$INDEX" ] || [ "$INDEX" == "null" ]; then echo false; else echo true; fi)
              - if [ $IS_PR != 'true' ]; then echo Skipping tests as branch not a pull request; fi
              # need to configure a dummy git user so that merge can proceed
              - if [ $IS_PR == 'true' ]; then git config user.email "p.eline@bitbucket.org"; fi
              - if [ $IS_PR == 'true' ]; then git config user.name "Phillipa Eline"; fi
              # perform merge of develop into PR branch (achieves the same code - but not history - as merging PR branch into develop)
              - if [ $IS_PR == 'true' ]; then GIT_MERGE_AUTOEDIT=no git merge --no-ff origin/develop; fi
              # run tests if we are on a PR branch
              - if [ $IS_PR == 'true' ]; then yarn test; fi
      caches:
        node-server: node_modules
    

    Shared credit for this with @minkydoodle

  14. Matt Ryall staff

    As part of this work, we should also consider adding:

    • ability to trigger builds (custom pipelines or manual steps) from the PR screen, per #13706
    • linking back to the related PRs from the Pipelines screen, per #13720.
  15. Oleg Zaimkin

    The solution provided by @Misha Wakerman for checking post-merge GIT_MERGE_AUTOEDIT=no git merge --no-ff origin/develop; does not seem to work anymore. Now repository clone step only pull one branch (despite there's no --single-branch switch).

    I tried several ways to fetch the develop branch but noone works.

  16. Luke Batchelor

    Hiya Oleg, If I'm understanding your issue correctly, I might have a solution. I messed around for ages trying to be able to do a couple of things around changing branches in builds. The key ended up being this snippet:

    # We fetch and checkout master here so that we have a local reference to "master" in other commands
    # (avoids the "ambiguous argument 'master': unknown revision or path not in the working tree" error)
    git fetch origin master
    git checkout master # (master doesn't exist until we do this checkout)
    git checkout - # checks out the previous ref
    

    repeating this for any other refs your need references to.

    Does that help?

  17. Rhys Laval

    Try git config --add remote.origin.fetch "+refs/heads/develop:refs/remotes/origin/develop" with a git fetch. This should correctly setup the develop branch for remote fetch

  18. Oleg Zaimkin

    Thanks for advising guys. The latter results in

    merge: origin/develop - not something we can merge
    

    The first results in:

    > git config remote.origin.fetch
    +refs/heads/develop:refs/remotes/origin/develop
    
    > git fetch --verbose https://${BB_AUTH_STRING}@bitbucket.org/${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}
    POST git-upload-pack (407 bytes)
    From https://bitbucket.org/winecity/portal-server
     * branch            HEAD       -> FETCH_HEAD
    

    It does work locally (fetches exactly what I need), but fails in pipelines.

    Update: Here's what I stopped at (succeeded):

                - if [ $IS_PR == 'true' ]; then git remote add upstream https://${BB_AUTH_STRING}@bitbucket.org/${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}; fi
                - if [ $IS_PR == 'true' ]; then git fetch --verbose upstream develop; fi
                - if [ $IS_PR == 'true' ]; then GIT_MERGE_AUTOEDIT=no git merge --no-ff upstream/develop; fi
    
  19. Mitch Lillie

    I was having false positives with the example from @Misha Wakerman so here is my take, using as many builtin Bitbucket env vars as possible.

    You only need to set two environment vars for the repo: USERNAME and APP_PASSWORD

    - JQ_QUERY=".values | map(.source.branch.name) | index(\"$BITBUCKET_BRANCH\")"
    - INDEX=$(curl --user $USERNAME:$APP_PASSWORD -s -XGET "https://api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_OWNER/$BITBUCKET_REPO_SLUG/pullrequests?q=state%20=%20%22OPEN%22&fields=%2A.source.branch.name&pagelen=50" | jq "$JQ_QUERY")
    - IS_PR=$(if [ -z "$INDEX" ] || [ "$INDEX" == "null" ]; then echo false; else echo true; fi)
    - if [ $IS_PR != 'true' ]; then echo Skipping tests as branch not a pull request; fi
    - if [ $IS_PR == 'true' ]; then npm test; fi
    
  20. Aneita Yang staff

    Hey everyone,

    Unfortunately still no progress on this request; however, I am interested in speaking with some customers about their requirements for this feature and to better understand why this is important. If you're interested in sharing your use case, please reach out to me at ayang@atlassian.com and we can schedule a quick 30min chat.

    Thanks!
    Aneita

  21. Eskild Diderichsen

    Here is how I wish it works.

    I want to:

    • trigger builds for pull requests against any branch
    • trigger builds for pull requests against a target branch (glob matching)
    • trigger builds for pull requests against the "main branch" (bitbucket repo option)
    • not trigger a build for pull requests that have merge error(s)
    • have the option to automatically trigger builds for subsequent commits pushed to a branch with an OPEN pull request, or trigger it manually on the PR page itself if auto-build is disabled
    • only build the last commit. See Travis' solution https://docs.travis-ci.com/user/customizing-the-build/#Building-only-the-latest-commit it explains it
    • build the pull request and/or the merged (simulated) pull request in an easy way. This is to see if the branch it self works (does it compile and tests are green) and if it still works when merged.
      • I want to configure different steps for the two builds. So on the pull request I want to run stuff like SonarQube, but on the merged pull request I only care about if it compiles and tests are green
  22. Gabriel de Marmiesse

    It would make sense to run the build at each PR and not at each push. I mean, it would be an easy save of building time. And I don't care about testing something in the feature branch until there is a PR.

  23. Anonymous

    I don't if it is enough, but, you can type on your commit [skip ci] and pipeline will not run. Independent of [skip ci] feature, I think that feature to add specific pipelines configuration it is important.

  24. Paul Brown

    Quite shocked to learn that Pipelines doesn't support triggering a build on creation of a PR, something that is a common development workflow practice and long supported by VSTS (and GitHub). I get that there might be workarounds using webhooks and calling Pipelines REST API or triggering the build on every push to the feature/task branch (which wastes minutes unnecessarily) but this really ought to be supported out of the box. After all, the point of a PR is to ensure only validated and healthy code is committed to a shared branch. Triggering a build on PR creation that also runs unit tests and performs code analysis is crucial to ensuring the code meets a minimum quality bar and if this fails, the PR should be blocked from being merged. Therefore, a swift resolution of this issue gets my vote!

  25. Robbert van Staveren

    Came across this thread while searching for PR building and Pipelines. A lot has been said already but I wanted to add that the lack of this functionality made me choose Jenkins over Pipelines. My company has most of the Atlassian products, and we made use of Pipelines before, but the unnecessary building of branches and the lack of being able to build PR's made us switch to Jenkins. I would prefer to use Pipelines, but not until this feature has been implemented.

  26. Josh Stevens

    I'm surprised this is still not a feature. Most CI pipelines I've worked with allow this. It could be as simple as allowing pipelines to be triggered via PR webhooks (which already exist). @Misha Wakerman has a good solution that seems to work, although the pipelines still only trigger on push events and not on PR creation. It also wastes ~20 seconds of build time for every branch push that is not a PR while it spins up the build agent and checks to see if it needs to build. This is not ideal with the limited amount of build time we have.

    The ideal pipeline workflow for our team would be to do a test merge, then run tests and build when a PR is created or update. Do nothing on non-PR branches. Merges to develop should simply build and push a docker image to our registry (tests were already run on the PR so they don't need to run again).

    As it stands right now, since @Misha Wakerman's solution doesn't trigger on PR creation, it actually proves to be wasteful since it generates a lot of ~20 second builds that do nothing, and we still have to manually trigger a build when we actually create a PR. Our best bet for now is probably to create a custom pipeline called "PR" and manually trigger it when we create PR's.

  27. Marcus Schumann

    I don't get why people want this, to be honest.

    Let's say you create a PR and have the pipeline run. If you get reviews on it and update the PR, the pipeline won't run in that case. Having pipeline trigger on commit is just better in every case since that's the actual code you want to test, not the PR itself?

    Now sure, I get the part where it's "wasting build minutes" while developing on the feature branch but then THAT should be the new feature, to only auto-trigger pipeline on commit when an active PR exists for the branch the commit is in.

  28. George Fekete

    A workaround could be to limit the push on the master branch for developers, so they will develop on feature branches. After the feature is finished, the developer can create a PR for the owner which will inspect it and merge it into the master branch, triggering the pipeline build.

    One drawback to this, that the pipeline will only be triggered after the merge commit.

    So the steps are: limit push on master (Settings -> Branch permissions) and allow PR merge for devs or for specific users only. Then in your bitbucket-pipelines.yml file limit the pipeline trigger only on the master branch.

  29. Paul Brown

    Marcus Schumann, auto-trigger pipeline on commit only when an active PR exists is exactly what's needed, because the pipeline needs to run if there is an active PR that has new commits pushed to the branch in response to a code review (if the PR is not abandoned), not just when the PR is first created. This could work as an option: trigger pipeline either when there's an active PR only, or on every commit. This keeps the current functionality (if people like it) but also covers the desire to only run the pipeline when there is a PR (and thus cut down on build minutes).

  30. Fraser Hardy

    The integration with Sonar is the main reason we need this. If a dev has not created a PR by time the pipeline run's for their last commit then the sonar scanner task skips over the task to add comments to the related open PR. We rely on an approval from Sonar to allow the PR to be merged, so currently the workaround is we have to just re-run the pipeline once the PR is created which is far from ideal.

    Also, as others have said above we encourage dev's to commit and push often so a subset of tasks on every commit then additional tasks on PR would be very useful.

  31. Noah Allen

    What's the holdup to implementing this feature? I can have a build run on other CI tools based on a PR in bitbucket because there are webhooks which can trigger events like this. Seems like the event system is kind of already in place on some level.

  32. Aneita Yang staff

    Hi everyone,

    No progress on this so far - the team are currently finishing up on some higher priority projects. However, this is on our medium term roadmap and we hope to have something to customers by the end of 2018.

    Thanks for your patience and thanks to everyone who's participated in a customer interview or provided feedback on this ticket.

    Aneita

  33. Taher Dhilawala

    Solution provided by @Misha Wakerman did not entirely worked for us. The only change required in his solution was to change the way we parse the response for open PRs from API.

     - INDEX=$(curl --user <$USERNAME>:<$APP_PASSWORD> -s -XGET "https://api.bitbucket.org/2.0/ repositories/<$TEAM>/<$REPO>/pullrequests?pagelen=50&state=OPEN" | jq ".values[] | .source.branch.name  | index(\"$BITBUCKET_BRANCH\")")
     - IS_PR=$(if [ -z "$INDEX" ] || [ "$INDEX" == "null" ]; then echo false; else echo true; fi)
    

    If there were multiple PRs open for other branches, the response we get was null\nnull - which means there are two open PRs where the index search with current branch fails. Since null\nnull is not equals to null, always the else condition would get executed setting IS_PR to true.

    To fix this, we had to change that part to,

    - INDEX=$(curl --user "$BITBUCKET_USER_NAME:$BITBUCKET_APP_PASSWORD" -s -XGET "https://api.bitbucket.org/2.0/repositories/<$TEAM>/<$REPO>/pullrequests?pagelen=50&state=OPEN" | jq ".values[] | .source.branch.name | index(\"$BITBUCKET_BRANCH\") | select(.)")
    - IS_PR=$(if [ -z "$INDEX" ]; then echo false; else echo true; fi)
    

    The key change was using select(.) which eliminates all the null values. So the next if condition only checks if IS_PR was set - if there was any open PR for the given branch, we will get an index otherwise it would be unset.

  34. gondo

    +1

    primary motivation: to save build minutes. and this is most likely the reason why it is not implemented yet, if ever. more build minutes -> more money for BB

    idea for a feature request: implement "step stop" command. we don't need it, but seeing the workaround scripts doing if [ $IS_PR == 'true' ]; then it might be beneficial to end test step asap

  35. mahesh gawali

    I wrote a simple python script to get around this,

    https://gist.github.com/maheshgawali/30058cd0ad2effadce247a8a1e2a634d

    just copy the script to your root path where your yml file is, rest of the usage is similar to the curl method described in the thread above.

    # $PR_CHECKER_USERNAME $PR_CHECKER_APP_PASSWORD export these using bitbucket pipeline environment variables
     - IS_PR=$(python bb-pr-checker.py $BITBUCKET_BRANCH $BITBUCKET_REPO_OWNER $BITBUCKET_REPO_SLUG $PR_CHECKER_USERNAME $PR_CHECKER_APP_PASSWORD)
     - if [ $IS_PR != 'True' ]; then echo Skipping tests as branch not a pull request; fi
     - if [ $IS_PR == 'True' ]; then
     - <do steps>
     - <do steps>
     - <do steps>
     - <do steps>
     - fi
    
  36. Log in to comment