Ability to restrict who can run deployment builds in Pipelines

Issue #13676 resolved
Gideon Koh
created an issue

When pipelines are deploying to production or have access to other important resources, customers want to either prevent changes to configuration or restrict the ability to run the pipeline to certain members of their team.

This is related to #12844, and needs further discussion with customers before we settle on a solution.

Original suggestion

Pipelines should only read from bitbucket-pipelines.yml that is in the main branch and run branch-specific tasks from that file as well. That way developers that do not or should not have access to the build file can be blocked from the main branch but still be able to push to other branches (which still runs the branch-specific build code defined in the bitbucket-pipelines.yml file in the main branch). bitbucket-pipelines.yml files in non-main branches should be ignored and, obviously, should not be required to be in the branch that is defining the branch-specific build code.

Official response

Comments (70)

  1. Matt Ryall staff

    Is this a customer request, Gideon? Do you have any more background on why the customer wants to prevent developers changing the build configuration? Thanks!

  2. James Robinson

    We have some external developers working on tasks for us but do not want them being able to potentially edit (intentionally or unintentionally) what happens with the server. If the bitbucket-pipelines.yml file could be restricted to only being read from the main branch we could deny access to that branch with branch permissions but the other branches defined in the file would still trigger the build process for those branches. That way the file could not be modified by unauthorized users but would still build the defined branches when commits are sent to those other branches.

  3. Matt Ryall staff

    Thanks for the clarification, James. It sounds like you want to prevent certain team members from making changes that affect how your server deployments work in Pipelines. Your proposal is to ignore branch-specific changes to bitbucket-pipelines.yml so the build configuration is not modifiable. Is that correct?

    At the moment, Pipelines is designed with configuration-as-code where the build configuration is intentionally versioned with the code. It is by design that different branches can have different build configurations, as is often necessary when the application code changes in a way that requires related build config changes. So at this point, we don't want to deviate from this design.

    However, in the future, we'd like to address features like #12844 by building some specific deployment features into Pipelines. At this point, we'd also starting considering improvements like limiting deployments to only certain users in the team, which would address your request more directly.

    I'd like to change this feature request to be more general as the "ability to restrict users who can deploy changes". Does that sound suitable to you?

  4. Greg Mascherino

    This is kind of a big deal now. Since you have given the ability to run any pipeline for any branch without limitation it is now possible for any developer to accidentally (or intentionally) run the master branch build on a feature branch. We use the Gitflow pattern and are using master as our prod deployment. What do I do to prevent running non-master code with the master pipeline?

  5. Matt Ryall staff

    Thanks for your votes and interest in this issue.

    This is on our medium term roadmap, and we're planning to have a solution available by the end of 2018.

  6. Thomas Beauvais

    This would be such an amazing feature!

    A little about our "ideal" use case, we have CI/CD for Dev, Stage and Prod.

    Dev is updated from commits to "master" in a typical CI/CD fashion.

    Stage step is manually executed by clicking "Run", which then tags and is ideally restricted to Leads.

    Prod step is manually executed by clicking "Run", restricted to a specific user's email/username.

    For Stage and Prod, we need to lock down the "Run" button to an email/user or role in Bitbucket.

    We can't even hack this since there is no knowledge in the Pipelines context via environment variables such as BITBUCKET_STEP_RUN_USER_EMAIL. This would be cool too since we could even do deployments on behalf of the user rather than an API key giving to Pipelines.

    Even better would be possibility to specify it in the pipelines.yml:

    pipelines:
      default:
        - step:
            name: Build, test and deploy to Dev
            image: node:8.6
            script:
              - npm install
              - npm test
              - npm run build
              - npm deploy --stage dev
        - step:
            name: Stage
            image: node:8.6
            trigger: manual
            group: Leads
            script:
              - npm deploy --stage stage
        - step:     
            name: Prod
            image: node:8.6
            trigger: manual
            email: xyz@foo.com
            script:
              - npm deploy --stage prod
    
  7. Pouya N

    @Matt Ryall thank you for your responses and like everyone else I am excited to see what the next version of Bitbucket Pipelines has to offer.

    I would like to throw in another idea that would work for our use case if I may.

    It would be interesting to see some sort of feature that would allow for Grouped Environment Variables. This would allow us to create and save grouped versions of our environment variables - specific to our actual environments and user groups. For example the developers who are given access to our Staging box are different then the ones who have access to our Development box, user roles are also much more loose between our production environment and our sandbox environments (and also within each of those realms).

    This feature would allow us to mimic a similar flow for access and an account administrator or a deployment manager would then have the ability to allow which user or user group would have access to a given Environment Variable Group (i.e staging, development, or whatever they’d like to name it) which would be needed for the scrip to execute to a certain box.

  8. Aneita Yang staff

    Hi everyone,

    Thanks for your feedback on this issue so far. We're currently investigating this feature request and would like to speak to some customers about their use cases and requirements for deployment permissions. If you're interested in seeing this feature and would like to discuss your use cases and requirements in an informal 30min chat, please send us an email at pipelines-feedback@atlassian.com so that we can schedule a convenient time to chat.

    I'm looking forward to hearing from you!

    Aneita

  9. Dominik Kohler

    I just found this. Proposed solution from Thomas Beauvais looks quite perfect to me. Maybe only improvement would be make group: and email: arrays, which then allows multiple groups/persons rather than one (you don't want to change the pipeline because that one person is sick, but there's an urgent fix that needs to be deployed).

    Out of the discussion, also it should be rather user: than email for single persons, which allows username or email-address.

    But these are small details, i would really appreciate the function in general.

  10. Lea Anthony

    For me, most of the problems stem from the fact that pipeline builds happen regardless of branch. A simple solution would be to have a pipelines setting that specifies which branches pipelines may run on, rather than the current "all". This allows us to take advantage of the protected branches model. Or maybe just not have it as part of the source code. Mixing application code and build configuration seems to appeal more to convenience than security.

  11. Luke Batchelor

    Off topic, but I'd slightly disagree @Lea Anthony , the most important thing is that there are controls over who can and cannot change build code and that there is a record of who changed what, when and who approved it. You can build that into whatever other controls you have over build configuration, or you can build it into the code and get it for free.

  12. Luke Batchelor

    Oh, definitely.

    A workaround for restricting to users btw would be to curl for your pipeline and check who created it (You'll need jq installed)

    - export CREATOR="$(curl -s "https://api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_SLUG/pipelines/$BITBUCKET_BUILD_NUMBER" | jq -r '.creator.username')"
    - test "$CREATOR" = "aui-team-bot" || (echo "$CREATOR does not have permission to run this pipeline. Exiting" && false)
    

    It will still start the pipeline, but at least a user wont be able to do any damage accidentally

  13. Luke Batchelor

    For sure, prevents accidents though.

    For us it's fine because we're trying to protect a custom build that can only run on master and only the bot account has the ability to merge to master.

  14. Caleb Cushing

    I have to say, I don't understand what the problem is restricting deploy's to a specific branch/tag is. Doesn't enterprise also allow restricting who can push certain branches or tags?

    options:
      docker: true
      max-time: 10
    image: gradle:jdk10
    pipelines:
      default:
      - step:
          name: gradle build
          caches:
          - gradle
          script:
          - gradle --version
          - gradle --stacktrace --info build
      tags:
        v*:
        - step:
            name: gradle publish
            caches:
            - gradle
            script:
            - gradle --version
            - export GRADLE_OPTS="--add-modules java.xml.bind '-Dorg.gradle.jvmargs=--add-modules java.xml.bind'"
            - gradle --stacktrace --info publish
    

    and if that's not good enough, then maybe have a separate repo that builds your docker container... and only give certain people to access to that repo.

    To me, the problem described here seems like one of a bad workflow. I ended up here looking at another issue, where I was going to propose project level environment variables, 'cause this wanted them per branch...

  15. Noah Matovu

    Hi Everyone, Please any updates and timeline on this issue regarding "adding the ability to create environment variables scoped to a particular deployment environment" Thank you.

  16. Adam Long

    We'd like to see access levels determine who can trigger builds regardless of branch permissions. Also access levels of who can schedule builds as that can be equally destructive. Currently anyone with write access can schedule a build to deploy to any environment at any time and no one will know about it until its probably too late.

    • role/group based permission extending Read/Write/Admin to include Build - for triggering builds
    • role/group based permission extending Read/Write/Admin to include Build - for scheduling builds
  17. Philip Hodder staff

    Hi everyone,

    We have released the first component of this feature, Deployment Variables.

    These will allow for you to define deployment specific variables that will only be available inside of the deployment step to that environment. e.g. Deployment keys. You can configure them in the Repository Settings -> Pipelines -> Deployments.

    The next step is to add user restrictions for running a deployment step. This will allow you to toggle if only a repository administrator can perform a deployment to a specific environment.

    This will be followed by branch restrictions, to ensure a deployment step can only performed from a specific branch.

    Thanks,

    Phil

  18. Ashvin Narayanan

    @Philip Hodder Thank you and the team for getting this feature underway. Looking at your update on what's available and the steps to follow, I am a bit concerned that perhaps the original requirement has been interpreted incorrectly. Could you please elaborate how using these new restrictions, the 'Original suggestion' mentioned by @gideon_koh can be accomplished? Bear in mind, that build code is not necessarily the same as deployment code. So on every push to any developer branch, I would like to run certain build code; this may or may not include code to perform an actual deployment. Could just be things like notifications to other members, or other CI tasks like running a test suite. The important thing is whatever I'd like to happen is only dictated by the yml in the master branch (or any other branch explicitly marked by the admin).

  19. Ashvin Narayanan

    Of course, this does not prevent the developer from modifying build scripts referenced from the yml file. If there are two scripts, run_on_master.sh and run_everywhere_else.sh for example, there's nothing stopping the developer from swapping contents of the two files. This creates a new requirement: perhaps it's not just the yml file that's ignored everywhere except on master (or any other branch explicitly marked by the admin), but an entire directory (let's call this directory devops). Now what does one call a directory of files (and perhaps other directories) that is managed by a group and changes can be versioned? Hope you see where this is going...

    The only real solution: Keep the application-code and DevOps repos separate. Azure DevOps (formerly VSTS) allows for this but the process is fairly convoluted.

  20. Caleb Cushing

    ^ what I said, keep them separate. The real feature needed is to have a successful build/step run be able to trigger a downstream repo, but invertedly. Like the ops repo can say it depends on the code repo, and then when the code repo successfully deploys its artifact to the artifact repository; that triggers a run of the ops repo.

    as a consultant, I'd really like to see env vars for bitbucket projects. Not for security reasons, but to prevent overlap (where 2 projects might share the same env var name, and each project might have multiple repos. For security reasons (accidental exposure) I would like to see a way to say these env vars are only for this step.

    oh, and a barely related annoyance, I'd like a way to bulk add env vars, like upload an env file, or just paste its contents instead of one painful field at a time.

  21. Philip Hodder staff

    Hello @Ashvin Narayanan @Caleb Cushing,

    We did some validation of our solution earlier on during the customer interviews in September and October.

    The use-cases we're currently covering in this ticket are:

    1. Anyone can run the Build Code.
    2. Only a select set of users can run a Deployment.
    3. A user is unable to perform a Deployment from the Build Code on a branch.

    This seems to cover what you're describing as well. If you think it isn’t covered, can you open another ticket with further context into the problem you’re trying to solve.

    We also have an existing ticket on triggering downstream builds, which you can also follow.

    Thanks,
    Phil

  22. Ashvin Narayanan

    @Philip Hodder Regarding the 3rd use case (A user is unable to perform a Deployment from the Build Code on a branch)...what would happen if a developer who is allowed to run a Deployment (as per use case 2), changes the yml file and commits it to a branch? Will the build run: a) using these branch-specific changes, or b) using the yml in the master branch?

  23. Caleb Cushing

    yeah, I don't think this covers any use case I'm worried about at all... my deployments are auto run on tags and can't/shouldn't be run with out a tag. I'll probably open another ticket for my use cases that pipelines doesn't currently handle. I still think that this is a bandaid on what is ultimately bad repo architecture. I would like to see a way to have variables that only apply to certain trigger for my deployments (need to checkout deployment variables and see if they can be used on my v* tag, but even then reading this sounds like they don't exist at the organization or project level, where I really need them.

  24. Paul O'Riordan

    @Philip Hodder

    Thanks for the update. This looks like a good improvement. It looks like the only problem preventing us from using this is the limited number of environments available through Bitbucket Deployments (qa, staging, production). Are there plans to make these configurable?

  25. Philip Hodder staff

    Hello everyone,

    I’m excited to announce that deployment restrictions are now available to all customers on our Premium plan!

    You can now configure admin and branch restrictions against each of your deployment environments, by going to the Deployment Settings, located in your: Repository Settings -> Pipelines -> Deployments.

    This integrates great with our existing Premium plan features, such as merge checks, to enhance the control and security of your repositories.

    You can learn more about configuring Deployments here: https://confluence.atlassian.com/bitbucket/bitbucket-deployments-940695276.html

    Thanks,
    Phil

  26. Ashvin Narayanan

    @Philip Hodder Before I invest in upgrading to premium, could you let me know if my use-case (as described previously) is achievable with this new feature?

    Use-case: All users are able to trigger builds as per the pipelines configuration set by admin, but only admin can update this configuration. The configuration includes not just the yml file but any external scripts that get executed from it.

    Motivation for above use-case: I want all users to be able to (say) trigger builds upon pull-request creation/update, but I only want admin to be able to update the actual config. This is so that (among other scenarios), a user cannot (as an example) update the script that gets called on pull-request creation to have a build deployed to production. Hopefully you can imagine the implications of that...

  27. Philip Hodder staff

    Sorry I missed your previous comment, @Ashvin Narayanan.

    We follow a similar workflow at Atlassian to meet compliance requirements, with some slight variations.

    For your use-case (I'll just do an example with production) I suggest you do the follow:

    1. Setup a branch restriction of doing a deployment to production, to only a specific branch (such as master). Then also configure a branch pipeline for branch to do the branch deployment. This ensures that the deployment can only happen on master. A branch pipeline will only ever run on the specified branch. So a change has to be written to the master branch in order to change the deployment configuration (see 2 for how to work around this).

    Example config:

    pipelines:
      branches:
        master:
          - step:
              name: Deploy to production
              deployment: production
              script:
                - ./run-deployment
        *:
          - step:
               name: Build and test
               script:
                 - ./run-tests
          - step:
               deployment: production
               name: Attempt to deploy to production
               script:
                 - # This step will immediately be stopped before execution as it's not running on the master branch. So even if a user tries to do a deployment on a branch with their own config, it will not run.
    

    Example branch restriction: Screen Shot 2019-03-14 at 5.07.17 pm.png

    2: Now to prevent anyone from editing the bitbucket-pipelines.yml on the master branch, you can use a branch restriction to enforce that only you (or other specific users/groups) can merge a pull request onto master, and enforce that no one (or only specific users) can write directly to the master branch.

    Example configuration: Screen Shot 2019-03-20 at 12.02.21 pm.png

    I'm not sure what you mean by "external scripts"? As in scripts sourced from outside of Bitbucket? If so, I suggest you use fixed/immutable versions of these scripts in your pipeline.

    Sorry for the wall of text. Do you need me to clarify anything here?

  28. Ashvin Narayanan

    @Philip Hodder By external scripts, I'm just referring to scripts you call from your pipelines yml file. In your example, this would be ./run-deployment. In my example below, it's run-tests.sh. These are version controlled within the same repo that the yml file resides in and can therefore be edited and pushed by any user (not just admin). Such files (along with the yml file itself) to me, constitute the overall ops configuration.

    Now as I've described in my use-case, I run a script on pull-request creation/update, as follows:

    pipelines:
      pull-requests:
        '**':
          - step:
              script:
                  - sh run-tests.sh
              services:
                  - docker
    

    Now as some non-admin user, I can edit run-tests.sh so that it uploads something to production, commit/push my changes to the branch I'm working in, and create a Pull Request to master.

    Question 1) Would this execute run-tests.sh in the branch that I'm trying to merge in or the one in master?<br/>

    Question 2) What would happen if as this same user, I make a change to the pipelines yml file itself? Will the pipeline run with my changes or the version that exists in master?

  29. Philip Hodder staff

    @Ashvin Narayanan

    Question 1: The script runs on the source branch, so the branch trying to merge into master.

    Question 2: The PR pipeline will run with the changes on the branch.

    I see what your concern is now.

    You'll need to use Deployment Variables configured for the production deployment environment, so that the step needs to be defined as a deployment step to production in order to get the deployment secrets, thus also applying the configured branch permissions. Without these variables, then you'll run into the issues you're worried about Assuming you have some form of auth controlling the actual deployment.

    So you'd want something like this:

    pipelines:
      branches:
        master:
          - step:
              name: Deploy to production
              deployment: production # Defining the deployment environment will inject the associated variables.
              script:
                - ./run-deployment $PRODUCTION_DEPLOYMENT_KEY # Variable only available on production deployment step.
      pull-requests:
        '**':
          - step:
              script:
                - sh run-tests.sh # Cannot access deployment variables as it's not a deployment step.
              services:
                - docker
          - step:
              name: Deploy to production
              deployment: production
              script:
                - ./run-deployment $PRODUCTION_DEPLOYMENT_KEY # Step will not start as production deployments can only be run on the master branch.
    
  30. Ashvin Narayanan

    @Philip Hodder While the solution you've provided may work when it comes to deploying, deployment is just one example of something you may want to perform as part of a build pipeline.

    How would you handle this scenario:

    pull-requests:
        '**':
          - step:
              script:
                - sh run-tests-and-auto-merge-if-coverage-sufficient.sh
                # Since the above needs to be performed on every branch, it follows that any API keys used is accessible on every branch.
              services:
                - docker
    

    What's to stop a user from modifying the sufficiency test within run-tests-and-auto-merge-if-coverage-sufficient.sh?

  31. Philip Hodder staff

    @Ashvin Narayanan, at the moment we're only focusing on the deployment use-case.

    For that PR example, the best option I can suggest is to use the general branch restrictions feature to require that only certain users can merge a pull request. That will prevent the user from merging code that shouldn't be updated onto the master branch (or other specified), but won't prevent them from modifying the files on their branch.

    At the moment there isn't a feature to have restrictions preventing a user from modifying a specific file.

    It's a valid use-case though, and I can understand why it's important to you.

    Can you open a feature request detailing your use-case. The last pull-request example you have stands out here. I think we may need to explore some different permission options, possibly allowing for all Pipelines configuration to be defined in a separate repository (which would then let you utilise repository-level permissions), or one of the suggested solutions of only using the master branch Pipelines definition for all builds (which we didn't use as the solution for the deployments use-case).

    I suggest you also follow this ticket, as we may come up with a solution that addresses your use-case: https://bitbucket.org/site/master/issues/14078/sharing-pipeline-yml-files

  32. Ashvin Narayanan

    @Philip Hodder Thanks, I'll open a new issue as you've suggested.

    As it stands, such security vulnerabilities prevent me from even having a pull-requests directive within my pipelines yml file, so hopefully the issue can be addressed soon so that I can actually start using that directive.

    Regarding the direction taken to just go down the deployment route...I'm not sure why this was done, as there exists a simple and more general solution, which follows almost immediately from what was mentioned in the original suggestion and which you've identified yourself: (always) using the master branch Pipelines definition for all builds (A).

    Of course the yml is not the only file that can be tampered with, there are other sensitive ones like I've shown (run-tests-and-auto-merge-if-coverage-sufficient.sh). All these files together constitute the ops configuration (B).

    Now (B) would imply the use of a separate repo just for ops (like I've myself suggested in a previous comment in this thread), but (A) can still be achieved by doing this very simple trick:

    Use a directory at the root of the code repo itself called .pipelines (for example). The yml file as well as any other sensitive files can go in here, and it's always read from the master branch.

    In short, this simulates a separate ops repo without actually needing one.

  33. Log in to comment