Allow more capable steps that can be reused

Issue #12751 open
Arjen Schwarz
created an issue

At the moment all the steps are limited to a list of commands that can be run. While useful, a more powerful solution would be to allow other types of scripts. An example of this would be an AWS Lambda step where you only have to provide your settings and everything else is already done for you. Currently you need to go through several steps, which includes adding a script to your repository, and that's not an optimal solution.

An example of how this works in a different CI tool is this step that I created for Lambda deployments. Especially the ability to share steps like this across your projects (or with other people), would make Pipelines even more powerful.

Obviously, this would depend on #12750

Official response

  • Matt Ryall staff

    Thanks for the feedback on this issue. We're starting early design work on this feature, with development due to start around March-April 2018, after a few higher priority items are completed.

    If you have some time to discuss your needs for reusable steps in Pipelines, please shoot me an email at mryall@atlassian.com, and we can line up a time to chat.

    Please note that there are several existing workarounds to reusing step logic in Pipelines that I'll note on this ticket in case people aren't aware of them:

    • write a Python/Ruby/Node/shell script in your repository, and call it from different places in your YAML configuration
    • create a custom Docker image, which can set up any necessary build prerequisites and bundle additional scripts
    • use YAML references to replicate some of the step logic in the same config file (painful but possible).

    None of these are complete solutions however, so that's what we'd like to deliver later this year -- something that supports reuse of bits of build configuration across repositories.

Comments (31)

  1. Julio César Canares García

    I got the issue were I need to run the test step in all branches and also having specific steps in some branches, something like in gitlab-ci

    stages:
      - build
      - test
      - release
    
    build:
      stage: build
      script:
        - echo foo
    
    unit-test:
      stage: test
      script:
        - echo foo
    
    code-quality-test:
      stage: test
      script:
        - echo foo
    
    release:
      stage: release
      script:
        - echo foo
      only:
        - master
    
  2. Jason Collinge

    Being able to share scripts between steps/branches is pretty important for making efficient and maintainable pipelines. For instance, the build steps for a staging and production branch need to be the same, but right now you have to duplicate code between the steps on both branches. This quickly gets long and ugly and increases the likelihood of having differences in the common steps creep in. Being able to share steps between different pipelines would also be useful as the number of pipelines grows (similar to templates in gocd) but I don't think it's as critical.

  3. Matt Ryall staff

    Thanks for the feedback on this issue. We're starting early design work on this feature, with development due to start around March-April 2018, after a few higher priority items are completed.

    If you have some time to discuss your needs for reusable steps in Pipelines, please shoot me an email at mryall@atlassian.com, and we can line up a time to chat.

    Please note that there are several existing workarounds to reusing step logic in Pipelines that I'll note on this ticket in case people aren't aware of them:

    • write a Python/Ruby/Node/shell script in your repository, and call it from different places in your YAML configuration
    • create a custom Docker image, which can set up any necessary build prerequisites and bundle additional scripts
    • use YAML references to replicate some of the step logic in the same config file (painful but possible).

    None of these are complete solutions however, so that's what we'd like to deliver later this year -- something that supports reuse of bits of build configuration across repositories.

  4. Danielo Rodriguez

    I'm also very interested on this. Currently our pipelines have too much repeated code, and having to change the setup involves changing all our pipelines, which is time consuming and error prone. We already npm packages that encapsulates the setup logic, but even for use the scripts there are some setup steps required that we are forced to repeat over and over again.

    All above workarounds involve some kind of duplication

  5. Terje Andersen

    +1 for interest. My team is considering the move from Jenkins, and we think Bitbucket pipelines seems promising. But to re-write steps that will be exactly similar across branch and tag pipelines seems like an hassle (not a complete show-stopper though, especially since you have some ideas on how to resolve this).

  6. Adam Copley

    Also considering the move from semaphoreCI. It would be good to be able to define "setup steps" that can be written once and run in any branch configuration's respective pipeline.

    Similarly, it would also be nice to have the option of whether or not to check out the repository again at the beginning of each build step to allow for nicer grouping of different tests without wasting build minutes on checking out again.

  7. Mina Luke

    I have the same issue. I am using different branches to utilize CI, staging, and production environments each of which has 2 steps and most of the scripts are the same so I want to share some of them.

  8. Jochen We

    hey guys. i had the same requirement. but since pipelines support yaml-aliases, you can use aliases to reuse steps. you can do somesthing like that:

    stepdefinitions:
      - builddeps: &builddeps
          name: install deps
          script:
            - dist/bitbucket/before_install.sh
            - dist/bitbucket/install.sh
          artifacts:
            - vendor/**
      - lintingtesting: &lintingtesting
          name: linting, testing
          script:
            - dist/bitbucket/lint.sh
            - dist/bitbucket/test.sh
    pipelines:
      branches:
        master:
          - step: *builddeps
          - step: *lintingtesting
      default:
        - step: *builddeps
        - step: *lintingtesting
    

    hope this helps.

  9. Mohammad Norouzi

    +1 This is really important as the steps are growing, the file becomes bulky and unreadable... plus it's easy to make unwanted mistakes because people are copying and pasting the shared steps

  10. Log in to comment