Load balancer upgrade scheduled Wednesday, Dec 8, 2am GMT for up to one hour

Bitbucket will be unavailable for up to one hour starting Wednesday, Dec 8th, 02:00 GMT while we upgrade our application architecture for high-availability and fail-over. We expect the downtime not to exceed one hour.

During this maintenance window we’ll deploy two new HAProxy load balancers clustered with Heartbeat. As it stands now we’re load balancing with Apache’s mod_rewrite RewriteMap feature. Repositories are statically distributed across our application servers. If your assigned server has any critical failures, we’re unable to redirect traffic to another server. Luckily that hasn’t happened often since migrating Bitbucket from EC2 to Contegix. With the new application architecture repositories will no longer be bound to a specific server. Should one of our application servers crash, one of the others in the load balancer pool will pick up the slack.

If you check out the footer on any Bitbucket repository page you can see it’s bound to a particular server:

After this change our new load balancers will pass requests to our application servers with the fewest connections and least load instead of a static mapping. In the footer you’ll see different servers handling your repository requests. An added benefit we’re looking forward to is the ability to remove our application servers from the load balancer pool without taking your repositories offline. Overall this will make Bitbucket more reliable and ensure less scheduled maintenance and downtime.