Install some self-hosted BitBucket Pipeline runners

[Dan Bonachea]

Our repo is set up to use BitBucket Pipelines on every push to the repository. Unfortunately the berkeleylab workspace has a very limited number of build minutes (600/year!), and when asked lab IT has made clear they won't be fixing that shortage any time soon. So we get nice Pipeline results on pushes to forks in our individual workspaces (which run under our own accounts that get a sane number of build minutes, eg 500/month), but we usually get no Pipeline results on pushes to our main repo.

BitBucket has recently added support for self-hosted runners that don't charge build minutes:

https://support.atlassian.com/bitbucket-cloud/docs/set-up-and-use-runners-for-linux/

IIUC we could follow these instructions to set this up within a docker container anywhere, and that would restore our capability to get automatic Pipeline results on every push to the main repo. .

[Dan Bonachea]

I've been poking this for awhile now and it doesn't look promising. Our goal was to automate self-hosted runner pipelines on the main repo and continue using automated Atlassian cloud runner on forks, but I don't think this is possible within the single yml configuration that must be shared by both.

BB pipelines provide a repo variable within steps ($BITBUCKET_REPO_FULL_NAME), but there doesn't appear to be any way to make a pipeline conditional on that value. Currently the only supported pipeline/step conditions are on branch/tag or file changesets, with no repo condition and no arbitrary expression allowing a variable. And since bitbucket-pipelines.yml lives in the repo, I don't see any way we could have that same config file use a self-hosted runner for our main repo and a cloud runner for our fork repos, without generating errors from each on every push that send spam email and seriously degrade our user experience.

If I set up the file for both types of runner then we get a "out of cloud minutes" error on every push to the main repo, and a "no self.hosted runner" error on every push to a fork. Here's an example of the pipeline fails on a single commit with two automatic pipelines, ones configured for cloud runner and the other is configured for self-hosted runner:

• fork (has cloud minutes, lacks self-hosted runner) vs
• main repo (lacks cloud minutes, has self-hosted runner)

Sadly it appears the BB pipeline status machinery is not smart enough to silently ignore automatic pipelines with no available runners, even when other automatic pipelines have a runner.

I even tried installing a self-hosted runner on my fork using our same local VM and repeatedly got an error. It appears bitbucket's runner container cannot coexist with multiple copies of itself on the same host system using different BB repos.

I guess the "Beta" status on this new pipeline feature refers to the lack of configuration controls necessary to setup anything non-trivial..

[Dan Bonachea]

Deferred until BitBucket Pipelines mature sufficiently to meet our requirements.

[Dan Bonachea]

From this week's Bitbucket "ChangeLog" blog:

Workspace Runners are now available NEW THIS WEEK

Runners allow you to run builds in Pipelines on your own infrastructure, and you won’t be charged for the build minutes used by your self-hosted runners. You can add runners to your pipelines in specific repositories or at the workspace level which applies those runners to all the repositories in that workspace.

So they've added workspace-level runners, but the self-hosted runner feature remains Beta and the available pipeline step conditions remain unchanged. This issue remains deferred until they add missing functionality we need.