Support parallel deployments

Issue #18754 duplicate
Barry Lagerweij created an issue

The current implementation for flexible deployments in Pipelines allows us to define multiple Test, Staging and Production deployments However, all deployments in a specific phase (e.g. Production) have to be executed sequentially. In our case, we have multiple production deployments for multiple countries (Kenya, Tanzania and Nigeria). With the current setup, we are forced to deploy to all production deployments sequentially (only one ‘Deploy’ button is clickable).

We would like to propose a change in the UI, where the user can trigger any manual deployment for a specific phase. (e.g. we would like the ability to deploy to Production-Nigeria without having to deploy Production-Kenya first).

Official response

  • Samuel Tannous staff

    Pipelines now supports parallel deployments steps!

          - parallel:
            - step:
                deployment: prod-east
            - step:
                deployment: prod-west
    

    All deployment steps in a parallel set must be for the same environment type (test, staging or production).

    Also only automatic deployment steps are possible at the moment as the manual trigger still isn’t allowed in a parallel set.

    I’ll keep this ticket open to track requests for manual and optional deployment steps.

Comments (13)

  1. Kevin Haggerty

    This would be a HUGE benefit to us as well - we’ve got 6 dev environments (yes, a bit crazy) and it would be nice to be able to deploy whatever we’ve got in production to any (or all) dev environment(s).

  2. Barry Lagerweij reporter

    For me, the most elegant solution would an extension to the bitbucket-pipelines.yml syntax, which allows us to use the ‘parallel’ keyword for deployments as well (currently the validations prevent you from using ‘parallel’ manual deployments.

    For instance:

    pipelines:
      branches:
        master:
          - step:
              script:
                - npm ci
                - npm run build
          - parallel:
            - step:
                name: Deploy Kenya
                deployment: prod-kenya
                trigger: manual
                script:
                  - npm run deploy:ke
            - step:
                name: Deploy Nigeria
                deployment: prod-nigeria
                trigger: manual
                script:
                  - npm run deploy:ng
    

    Using the above syntax would make the YAML backwards compatible (keep current behavior for sequential steps), and allows us to specify where we need multiple parallel manual deployments. It would also allow you to configure the dependencies correctly: If you require that all ‘staging' steps have finished before you’re allowed to deploy to prod, you can use the following:

          - parallel:
            - step:
                deployment: staging-kenya
            - step:
                deployment: staging-nigeria
          - parallel:
            - step:
                deployment: prod-kenya
            - step:
                deployment: prod-nigeria
    

    Whereas the following would allow you to deploy to ‘prod-kenya’ while the nigeria pipelines have not executed: (but ‘staging-kenya’ was completed)

          - parallel:
            - step:
                deployment: staging-kenya
            - step:
                deployment: prod-kenya
          - parallel:
            - step:
                deployment: staging-nigeria
            - step:
                deployment: prod-nigeria
    

    In the above examples I’m assuming that ‘staging-*’ deployments are configured in the ‘Staging’ phase, and prod-* deployments are production. I hope this makes sense. This is just a suggestion, at this.moment I’m open to any solution that would allow us to deploy to multiple environments in any random order. (a quick fix might be to add a new trigger type called ‘manual-parallel’, which will always make the deploy button enabled)

  3. Samuel Tannous staff

    Pipelines now supports parallel deployments steps!

          - parallel:
            - step:
                deployment: prod-east
            - step:
                deployment: prod-west
    

    All deployment steps in a parallel set must be for the same environment type (test, staging or production).

    Also only automatic deployment steps are possible at the moment as the manual trigger still isn’t allowed in a parallel set.

    I’ll keep this ticket open to track requests for manual and optional deployment steps.

  4. Kevin Haggerty

    Great news that parallel deployment steps are now a thing!

    The restriction that it’s only for automatic deployment (no manual trigger) means it’s still not useful for us unfortunately, but I’m hopeful the additional options will be available soon - specifically the manual trigger and optional steps.

  5. Max Heyer

    Hi @stannous,

    in our case, we need a feature to deploy the production environment without deploying the dev environment first. In your solution, it’s possible to run deployment in parallel. As described in the ticket request, it should be possible to run any deployment environment without running the previous deployments.

  6. Barry Lagerweij reporter

    Thanks for the update, great to finally see some progress. Unfortunately, our product-owners demand the freedom to manually deploy to a specific environment, so the automatic parallel steps are not suitable for us. @Samuel Tannous any indicative timelines for the manual steps? Are we talking weeks or months? We are considering moving to AWS CodeBuild and Code Pipelines (which do support this feature) for our deployments, but I would prefer to stay with Bitbucket for deployments, since I really like the integrated UI.

    Can we merge issue https://bitbucket.org/site/master/issues/16650/manual-trigger-for-steps-running-in and this issue? They appear to be duplicates…

    @Max Heyer I think it’s generally good practice (and in line with how artifact repositories elevate binaries) to first run through Test, then Staging and finally Production.

  7. Aneita Yang staff

    Hey everyone,

    Thanks for the feedback regarding the need for manual/optional deployment steps within a parallel group.

    We decided to support parallel steps (without manual/optional deployment support) for the first version as it was a quick win and would cover some of the use cases that customers had for parallel deployments. We recognise that this isn’t sufficient for some teams out there. Given our current priorities, however, this is unlikely to be something that we do within the next few months.

    Issue #16650 is an open issue tracking the request to support manual steps in a parallel group, so I’ll close this issue as a duplicate. If being able to run deployments optionally/manually in a parallel group is something that you’re interested in, I encourage you to vote for and watch issue #16650 for updates.

    Aneita

  8. Log in to comment