Support BitBucket Pipelines

[Roman Semenov]

Show BitBucket build status notification, similar to TeamCity integration.

API reference: https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/pipelines

[Daniil Penkin]

Hi @poma,

Thank you for trying out Linky and sharing your idea of improvement!

Indeed, Pipelines build status would be a nice feature in IDE. I've added it to my feature backlog, so in one of the next versions I'll look into implementing it. I'm currently working on in-IDE PR reviews, and build status is likely to be the next feature.

Cheers, Daniil

[Daniil Penkin]

Not yet exactly what this ticket is about, but I started working on integration with Bitbucket Pipelines.

Version 8.0.101 adds support for the bitbucket-pipelines.yml configuration file: validation, completion, navigation and some inspections. The next step is to show build statuses

[Gareth Jones]

@Daniil Penkin loving the new pipelines file support!

I made a schema file for the configuration about a month or so ago, which seems to be about equal with yours; the only difference I can find is that IntelliJ only seems to provide autocompletino & such when using my schema file.

I'll attach a copy here, if you'd like to see if there's anything useful in it :)

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "definitions": {
    "image": {
      "type": ["string", "object"],
      "required": ["name"],
      "properties": {
        "name": { "type": "string" },
        "username": { "type": "string" },
        "password": { "type": "string" },
        "email": { "type": "string" },
        "run-as-user": { "type": "number" },
        "aws": {
          "type": "object",
          "required": ["access-key", "secret-key"],
          "properties": {
            "access-key": { "type": "string" },
            "secret-key": { "type": "string" }
          },
          "additionalProperties": false
        }
      },
      "additionalProperties": false
    },
    "strings": {
      "type": "array",
      "items": { "type": "string" },
      "uniqueItems": true
    },
    "customVariable": {
      "type": "object",
      "required": ["name"],
      "properties": {
        "name": {
          "type": "string"
        }
      }
    },
    "customVariables": {
      "properties": {
        "variables": {
          "type": "array",
          "items": { "$ref": "#/definitions/customVariable" },
          "uniqueItems": true
        }
      },
      "additionalProperties": false
    },
    "max-time": {
      "type": "number",
      "description": "The maximum amount of minutes a step can execute for.",
      "minimum": 0,
      "default": 120,
      "maximum": 120
    },
    "clone": {
      "type": "object",
      "required": [],
      "description": "Contains settings for when we clone your repository into a container",
      "properties": {
        "lfs": {
          "type": "boolean",
          "default": false
        },
        "depth": {
          "type": "number",
          "minimum": 0,
          "default": 50
        }
      },
      "additionalProperties": false
    },
    "trigger": {
      "type": "string",
      "enum": ["automatic", "manual"]
    },
    "size": {
      "type": "string",
      "enum": ["1x", "2x"]
    },
    "script": {
      "type": "array",
      "items": {
        "oneOf": [{ "type": "string" }, { "$ref": "#/definitions/pipe" }]
      }
    },
    "deployment": {
      "type": "string"
    },
    "pipe": {
      "type": "object",
      "required": ["pipe"],
      "properties": {
        "pipe": {
          "type": "string"
        },
        "variables": {
          "type": "object",
          "additionalProperties": true
        }
      }
    },
    "stepInner": {
      "required": ["name"],
      "properties": {
        "name": { "type": "string" },
        "image": { "$ref": "#/definitions/image" },
        "deployment": { "$ref": "#/definitions/deployment" },
        "services": { "$ref": "#/definitions/strings", "maxItems": 5 },
        "caches": { "$ref": "#/definitions/strings" },
        "size": { "$ref": "#/definitions/size" },
        "script": { "$ref": "#/definitions/script" },
        "after-script": { "$ref": "#/definitions/script" },
        "artifacts": { "$ref": "#/definitions/strings" }
      },
      "additionalProperties": false
    },
    "step": {
      "type": "object",
      "required": ["step"],
      "properties": {
        "step": {
          "allOf": [
            {"$ref": "#/definitions/stepInner"},
            {
              "properties": {
                "trigger": { "$ref": "#/definitions/trigger" }
              }
            }
          ]
        }
      },
      "additionalProperties": false
    },
    "stepParallel": {
      "type": "object",
      "required": ["step"],
      "properties": {
        "step": { "$ref": "#/definitions/stepInner" }
      },
      "additionalProperties": false
    },
    "steps": {
      "type": "array",
      "items": { "$ref": "#/definitions/step" },
      "additionalProperties": false
    },
    "stepsParallel": {
      "type": "array",
      "items": { "$ref": "#/definitions/stepParallel" },
      "additionalProperties": false
    },
    "parallel": {
      "type": "object",
      "required": ["parallel"],
      "properties": {
        "parallel": { "$ref": "#/definitions/stepsParallel" }
      },
      "additionalProperties": false
    },
    "pipelines": {
      "type": "object",
      "required": [],
      "additionalProperties": { "$ref": "#/definitions/pipeline" }
    },
    "pipeline": {
      "type": "array",
      "items": {
        "oneOf": [
          { "$ref": "#/definitions/step" },
          { "$ref": "#/definitions/parallel" }
        ]
      }
    }
  },
  "required": ["image", "pipelines"],
  "properties": {
    "$schema": { "type": "string" },
    "image": { "$ref": "#/definitions/image" },
    "clone": { "$ref": "#/definitions/clone" },
    "options": {
      "type": "object",
      "description": "Contains global settings that apply to all your pipelines.",
      "required": [],
      "properties": {
        "max-time": { "$ref": "#/definitions/max-time" }
      },
      "additionalProperties": false
    },
    "definitions": {
      "type": "object",
      "properties": {
        "caches": {
          "type": "object",
          "required": [],
          "additionalProperties": {
            "type": "string"
          }
        },
        "services": {
          "type": "object",
          "required": [],
          "additionalProperties": {
            "type": "object",
            "required": ["image"],
            "properties": {
              "image": { "$ref": "#/definitions/image" },
              "memory": {
                "type": "number",
                "minimum": 128,
                "default": 1024,
                "maximum": 7128
              },
              "variables": {
                "type": "object",
                "additionalProperties": true
              }
            }
          }
        },
        "steps": { "$ref": "#/definitions/steps" },
        "additionalProperties": false
      },
      "additionalProperties": false
    },
    "pipelines": {
      "type": "object",
      "required": ["default"],
      "properties": {
        "default": { "$ref": "#/definitions/pipeline" },
        "branches": { "$ref": "#/definitions/pipelines" },
        "tags": { "$ref": "#/definitions/pipelines" },
        "bookmarks": { "$ref": "#/definitions/pipelines" },
        "pull-requests": { "$ref": "#/definitions/pipelines" },
        "custom": {
          "type": "object",
          "required": [],
          "additionalProperties": {
            "type": "array",
            "items": [
              {
                "oneOf": [
                  { "$ref": "#/definitions/customVariables" },
                  { "$ref": "#/definitions/step" }
                ]
              }
            ],
            "additionalItems": {
              "oneOf": [
                { "$ref": "#/definitions/step" },
                { "$ref": "#/definitions/parallel" }
              ]
            }
          }
        }
      },
      "additionalProperties": false
    }
  },
  "additionalProperties": false
}

[Daniil Penkin]

Hi @Gareth Jones ,

Huge thanks for posting your schema! I’ll see if it has anything that I missed.

IntelliJ only seems to provide autocompletion & such when using my schema file.

Hm, this doesn’t sound right. The whole point of providing a schema was to add basic autocompletion and validation Both work fine for me with the schema bundled with Linky.

Can you double check that the right schema is selected when you edit the bitbucket-pipelines.yml file?

If everything looks correct, but the autocompletion still doesn’t work, can you post an exact example of such file and place where you tried to invoke completion?

Cheers,
Daniil

[Gareth Jones]

Sorry I left out an important part: step in a definition.

Using your schema I don't get auto-completion in steps within definitions in this pipelines configuration:

image: node:12.10-alpine

clone:
  depth: 3

definitions:
  caches:
    npm: ~/.npm
  steps:
    - step: &install-and-test
        name: 🧪 Install and Test
        caches:
          - npm
        script:
          - npm ci --ignore-scripts --no-audit
          - npm audit
          - npm run lint
          - npm test -- --coverage
    - step: &prep-for-deployment
        name: 🗳️ Prepare for Deployement
        image: atlassian/pipelines-awscli
        script:
          - ci-scripts/build-variables
        artifacts:
          - ci-variables
    - step: &build-container
        name: 📦 Build Container
        image: atlassian/pipelines-awscli
        services:
          - docker
        caches:
          - docker
        script:
          - source "ci-variables"

          - eval "$(aws ecr get-login --no-include-email --region "$AWS_REGION")"

          - docker build --pull -t "$ECR_URL:$BUILD_TAG" .
          - docker push "$ECR_URL:$BUILD_TAG"
        artifacts:
          - ci-variables
    - step: &create-task-definition
        name: 📋 Create Task Definitions
        image: loicmahieu/alpine-envsubst
        script:
          - source "ci-variables"
          - envsubst < infrastructure/task-definition.json > definition.json
        artifacts:
          - ci-variables
          - definition.json
    - step: &deploy-definition-to-cluster
        name: 🚀 Deploy Definition
        image: atlassian/pipelines-awscli
        trigger: manual
        script:
          - source "ci-variables"

          - >
            TASK_DEFINITION_ARN=$(
              aws ecs register-task-definition \
                --cli-input-json file://definition.json \
                --output text \
                --query "taskDefinition.taskDefinitionArn"
            )
          - echo "Created new Task Definition '$TASK_DEFINITION_ARN'"
          - >
            aws ecs update-service \
              --force-new-deployment \
              --cluster "$ECS_CLUSTER_ARN" \
              --service "$ECS_SERVICE_ARN_PREFIX/$APP_SERVICE_NAME" \
              --task-definition "$TASK_DEFINITION_ARN"
        artifacts:
          - ci-variables
          - definition.json
    - step: &echo-variables
        name: 🖨️ Echo Variables
        script:
          - cat ci-variables
          - source ci-variables
          - echo "$ECR_URL:$BUILD_TAG"
    - step: &test-aws-creds
        name: 🤔️ Test AWS Creds
        image: atlassian/pipelines-awscli
        script:
          - source ci-variables
          - >
            eval "$(
              aws ecr get-login
              --no-include-email
              --region "$AWS_REGION"
            )"
        artifacts:
          - ci-variables

pipelines:
  default:
    - step: *install-and-test
  branches:
    master:
      - step: *install-and-test
      - step:
          <<: *prep-for-deployment
          deployment: staging
      - parallel:
          - step: *build-container
          - step: *create-task-definition
      - step: *deploy-definition-to-cluster

I can confirm I'm using the correct schema :)

[Daniil Penkin]

Ah yeah, this indeed doesn’t work in the current version. The reason for this is that definitions is not limited to defining steps which then can be reused. Basically, any element in the yaml can be aliased, and it’s just a convention that you put common bits inside definitions. This also means that "additionalProperties": false you set for properties under definitions is generally incorrect: take a look at Linky’s configuration for instance – it has build and deploy.

definitions.steps element that you defined in your schema is another good convention but it is not enforced by Pipelines, that is if I added it to Linky, it might’ve shown someone’s schema as invalid, although it is accepted and works perfectly in Pipelines. I mean the case when someone defined definitions.steps but used different internal structure for that element.

Similarly, Linky’s schema allows any extra top-level elements because some people don’t follow the convention and simply keep their aliases at the root of the document.

With all that said, I totally agree that auto-completion should ideally work anywhere. JSON schema was an easy win since it provides both completion and validation, but it obviously doesn’t cover all use cases. So I decided take iterations on this and keep the JSON schema close to the actual Pipelines validation algorithm. Completion for configurations with aliases is in my TODO list for the future releases. I have two ideas around this but need to investigate if they’re feasible in IntelliJ. Will see how this pans out

I hope this makes sense. I’m open to any ideas, please share your thoughts.

[Daniil Penkin]

Removing version: 3.4 (automated comment)