Cache Cloned and checkout repository in Pipelines Multi step

Issue #15245 wontfix
Pravin Mahendran
staff created an issue

In pipelines multi step, each step will run a "Build Setup" which will clone and checkout the repository. In scenarios, while the first step is in progress, the branch is deleted. The following step will fail at the "Build Setup" as the clone will fail due to the branch does not exist any longer.

It would be good if we can cache the initial clone & checkout to be used in subsequent steps. This could also help with the duration of the build.

Comments (3)

  1. Matt Ryall staff

    Thanks for creating this ticket, Pravin.

    I'd like to understand the use case for deleting the branch before the build is complete. Teams shouldn't be merging changes and deleting branches before the build passes. If they're branching for releases/deployments via manual steps, I also wouldn't expect them to delete the branch, as they'd want a record of the release done on that branch.

    At the moment, this is the expected and designed behaviour of multiple steps in Pipelines. They are multiple distinct containers, so you can use different Docker images in each. If you want to run multiple commands in the same container, then you can do so within a single step.

    We looked at the idea of copying the clone between steps, but it was in fact slower. Because we want to be able to resume steps at any point in the next week (to support manual steps), any data passed between steps needs to be on persistent storage (S3 for us). Uploading data to S3 between steps is costly and time-consuming, and resulted in the pipeline running slower than if we cloned from Bitbucket at the start of each step. So we minimise the data passed between steps to just the declared artifacts.

    Unfortunately for the moment, handling branch deletion during the middle of a multi-step pipeline is something we won't be able to cater for. The customer will need to look at deleting the branches once all the builds are complete, if they want to avoid build failures.

  2. Log in to comment