Allow users to define tasks / step configuration which can be reused

Issue #17181 resolved
Aneita Yang
staff created an issue

It's common for users to have Pipelines configured across multiple repositories. Sometimes, pipelines across multiple repositories have the same step configuration.

Simplify a user's Pipelines configuration experience by allowing them to define sections of configuration which can be reused.

This issue has been split out from #12751.

Comments (4)

  1. Aneita Yang staff reporter
    • changed status to open

    Including Raul's comment from #12751 for reference:

    Some good news - we've started speccing out a feature that will let you share scripts between multiple repositories. This will help reduce the amount of repeated configuration across your repositories and make your bitbucket-pipelines.yml file more concise. We'd love to understand whether this is something that will suit the needs of your team:

    • You can share scripts across repositories by creating a task.
    • Tasks are built on Docker which gives you the flexibility to use any language or CLI to define your task, and also has the benefits of isolation (tasks won't affect the execution of another task) and reproducibility.
    • Tasks are defined in separate repositories which helps provide versioning capabilities.
    • To define a task, you will provide a YAML file with information about the task including:
      • name
      • description
      • base Docker image
      • parameters that are required by the script
      • script commands.
    • We will use Pipelines in the task repository to automatically build a Docker image for your task and push it to your specified Docker registry.
    • You can use tasks in your bitbucket-pipelines.yml file by providing account and repo name (account/repository) as the task ID, version number and the parameters required by the task (passed as environment variables).

    An example of how you might use a task looks like this (keep in mind that this is just an example and that the syntax has not yet been finalised):

    pipelines:
      branches:
        master:
          - step:
              name: Deploy to test
              deployment: test
              script:
                - echo "Starting deployment"
                - task: account/my-deploy-task:1.0
                  name: Deploy docker image to test environment
                  parameters:
                    IMAGE: my-image:latest
                    USERNAME: $SERVER_USER
                    PASSWORD: $SERVER_PASS
                - echo "Finish deployment"
    

    This model also allows us to provide a set of supported tasks which will simplify the configuration for using Pipelines with other tools (e.g. AWS Lambda, AWS S3, Kubernetes, etc.).

    As mentioned, this solution allows you to define and share scripts between repositories. If you're interested in reusing configuration within the same bitbucket-pipelines.yml file, you can use YAML references to do this today. We'd love your feedback on our proposed solution and to understand whether it suits the use cases described on this issue - you can comment on this issue or send us an email at pipelines-feedback@atlassian.com with your thoughts.

    Thanks for helping us improve Pipelines!

  2. Dushyant Chillale

    Hi Aneita,

    Thanks a lot for creating this issue after collating all the requirements from various users across multiple threads!

    I'm interested in this feature as well and I'd like to know if there is any update on this.

  3. Aneita Yang staff reporter

    Hi @Dushyant Chillale,

    Thanks for reaching out. We're currently working on providing an inbuilt set of templates to help users get started with Pipelines and to simplify their configuration experience. As part of that work, we'll be looking at extending the model so that you can write your own templates and reusable script definitions (similar to the approach detailed in the first comment). This is something that we're hoping to have available to customers by the middle of 2019.

    I'll be updating this ticket with our progress so please continue to watch this ticket for updates.

    Thanks,
    Aneita

  4. Aneita Yang staff reporter

    Hi everyone,

    Good news - we've just shipped Bitbucket Pipes which make it easier to build powerful, automated CI/CD workflows in a plug and play fashion without the hassle of managing integrations. We’ve worked with industry leaders including Microsoft, AWS, Slack and more to build over 30 supported pipes that help automate your CI/CD pipeline. On top of this, we've also made it simple to create your own pipe to help abstract any duplicated configuration across your repositories.

    In a nutshell, a pipe is a custom Docker image for a container, which contains a script to perform a task. A pipeline calls this pipe, passing it some variables and the magic happens! A pipe is made up of a few different files:

    1. a script, or binary, the code that performs the task
    2. a Dockerfile, which tells us how to build the Docker container that runs your script
    3. (optional) metadata and readme docs, to make your pipe easy to understand
    4. (optional) some CI/CD configuration so that you can easily update the pipe

    For more information on how to write your own pipe for use across repositories, check out our documentation. Note that if you're looking to reuse configuration within the same bitbucket-pipelines.yml file, you can also use YML anchors.

    The full list of supported pipes can also be found here.

    We hope you enjoy this addition to Pipelines!

    Aneita

  5. Log in to comment