Support YAML merge format in `bitbucket-pipelines.yml`

Issue #15972 open
Pavel Savchenko
created an issue

So that we can write complex pipelines while keeping the file clean and readable.

Supported in YAML 1.1, docs: http://yaml.org/type/merge.html

Example:

image: user/my-image:latest

x-run-tests:
  &run-tests
    name: Run tests
    services:
      - docker
    script:
      - ./test.sh

x-deploy-to-staging:
  &deploy-to-staging
    name: Deploy to staging
    deployment: staging
    trigger: manual
    script:
      - ./deploy.sh user@staging.com

pipelines:
  default:
    - step:
        <<: *run-tests

    - step:
        <<: *deploy-to-staging

  branches:
    develop:
      - step:
          <<: *run-tests

      - step:
          <<: *deploy-to-staging
          trigger: auto

    master:
      - step:
          <<: *run-tests
      - step:
          name: Deploy to production
          deployment: production
          trigger: manual
          script:
            - ./deploy.sh user@example.com

Comments (6)

  1. Matt Ryall staff

    Hey Pavel,

    YAML extensions actually work in Pipelines, but the YAML syntax in your example isn't quite correct so they're not working for you. To fix it, you should just need to outdent the step examples one level:

    x-run-tests: &run-tests
      name: Run tests
      services:
        - docker
      script:
        - ./test.sh
    
    pipelines:
      default:
        - step:
            <<: *run-tests
    

    There are two caveats with using YAML references in Pipelines:

    1. Our YAML validator in the Bitbucket UI doesn't currently support them. So you need to commit these advanced configurations from the command line. Unfortunately this almost makes them a bit painful to test. We'll look into fixing that.
    2. The error messages in the product are not very helpful for debugging YAML problems. So it can be easy to mess up the indenting, or reference an anchor before you declare it (something I didn't realise until researching this issue), and the error message you get isn't helpful at all.

    I'm going to write some documentation on how to do this shortly, because the question has come up a few times. In my example, I'm going to suggest putting reusable steps under the definitions section, and only using extensions where needed, otherwise simple references work.

    So something like this is what we'll recommend:

    definitions:
      steps:
        - step: &build-test
            name: Build and test
            script:
              - mvn package
            artifacts:
              - target/**
        - step: &deploy
            name: Deploy
            deployment: test
            script:
              - ./deploy.sh target/my-app.jar
    
    pipelines:
      branches:
        develop:
          - step: *build-test
          - step: *deploy
        master:
          - step: *build-test
          - step:
              <<: *deploy
              deployment: production
              trigger: manual
    

    Hope that helps. Please let us know how you get on.

    Regards,
    Matt

  2. Janne Nykänen

    YAML references just stopped working as of today? Is this a permanent thing or maybe some temporary glitch? I have been removing YAML references from our builds the whole day...

  3. Matt Ryall staff

    Sorry about that, we shipped a breaking change in our parser configuration. Should now be fixed.

    We don’t have any tests in place for this currently, but will rectify that shortly.

  4. Log in to comment