Bitbucket Pipelines provides a caching feature that provides the ability to cache external build dependencies and directories e.g. 3rd-party libraries. This means faster builds and fewer consumed build minutes.
However, caches currently live for 7 days and are invalidated at the end of that period. And updates to build dependencies do not cause the cache to be refreshed which means that builds can sometimes use old versions of dependencies, leading to downstream errors. We have this highly voted suggestion where customers are looking for an option to invalidate and refresh the cache automatically whenever there is a change in the build dependency.
In this blog post, we are sharing a solution that can help to invalidate and refresh the cache automatically whenever there is a change in the build dependency.
Here’s how it works:
We use the
atlassian/bitbucket-clear-cache pipe to delete the cache in the pipeline. Here’s the yaml for this pipe.
pipe: atlassian/bitbucket-clear-cache:3.1.1 variables: BITBUCKET_USERNAME: $BITBUCKET_USERNAME BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD CACHES: [ "node"]
Next, the condition changeset is used to execute a step only if there is a change in a specific file. If not, the step will be skipped.
We will add the changeset condition to indicate the file to be verified for the changes before actually deleting the cache. The structure in your YAML file will be something like this:
pipelines: default: - step: name: delete cache if changes in the build dependencies script: - pipe: atlassian/bitbucket-clear-cache:3.1.1 variables: BITBUCKET_USERNAME: $BITBUCKET_USER_NAME BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD CACHES: ["node"] condition: changesets: includePaths: - package.json - step: name: build and create/reuse the cache caches: - node script: - npm install - npm test
That’s it. This process will check your dependencies for changes, and if there is a change, it will delete the existing cache and create a new cache so your builds are always using the most current versions of dependencies.
- Caches are currently global and shared across multiple branches. When the cache is invalidated it will clear the cache for all branches, until it is cached again.
- Changesets only find files in the last commit, so if multiple commits are pushed, only the last one will be checked for the includePaths section.
Watch and share your feedback in BCLOUD-16314 feature request as we will look into implementing out of the box support and better caching / invalidation strategies in the future.