Overview

bitbucket-color-diff

Uses Google App Engine (with Pygments) to send colored diff emails on commit events in Bitbucket repositories.

Alternatively, you can use this on your local server (with Django or similar). This code is not available here yet (you are welcome to jump in and add it for me).

image

Note: HTML layout was "inspired" in the Perl's SVN::Notify (which I missed from back in the days with Subversion).

Installation

Google App Engine

Basically, you will need to have a Google App Engine ready to be deployed. If you don't know how to do this, read their official docs.

Once you downloaded this code, you will need:

(1) Add 3rd-party modules

Google App Engine by default does not come with Pygments library, so you need to add it by yourself. Additionally, I added the Chardet module to help fixing encoding issues (see more details on this in Issue #1).

On linux:

For convenience, I have created a script (for Linux), so you can just execute:

    $ gae/get_3rd_party_modules.sh

And the sources will be placed as required by this project to run properly in GAE.

(2) Update the code

You will need:

a. Add your app name

Go to gae/app.yaml file and place your app name there.

    ...
    application: ### YOUR APP NAME HERE ###
    ...

You can create your app here, or read more details about it here.

b. Configure Settings

Go to gae/settings.py file and edit all applicable settings:

  • Bitbucket credentials - only if it is a private repository (it needs to be a valid Bitbucket user with access priviletes to your repository)
  • Internal credentials (to rely on Basic Authentication from BitBucket requests) - read more about this here (this is to ensure that no external requests will access this script on Google, even though they are public - this is recommended, as Google does not provide any simple/basic auth features)

c. Update the HTML/TXT

Optionally, you can edit the HTML or TXT yourself. Look for the files gae/tpl/email.html and gae/tpl/email.txt (so you can update the footer or anything else to fit your needs).

(3) Upload to Google App Engine

    $ appcfg.py upload gae/

(4) Set up POST service in your Bitbucket repository

Go to your repository settings, then Services, and add the POST service pointing to your Google App engine application URL:

    # this URL will send to me@example.com
    # where the username and password are the internal credentials
    # you define in the settings.py file.
    https://username:password@your-app-id.appspot.com/commit/me@example.com

That's it!

Note: The code is running on secure always mode, so the URLs should be HTTPS formatted (better security).

Further Notes

Google App Engine Limitation pain

This was supposed to be a minimalistic approach to render a colorized diff for commits in BitBucket. As it turns out, Google pushes harsh limitations, especially to a billing disabled (free) account:

  • 30s deadline (for any requests)
  • 5 min async task deadline
  • task data exchange limits (must rely on datastore instead)
  • 340KB email size limit
  • 100 emails per day
  • 128MB RAM for intance class F1

It basically depends on what you want to accomplish and how bad you want to have the logs delivered to you by email.

The code has been updated to deal mas much as possible with such limitations:

  • commit callback creates a task for raw DIFF fetching
  • the diff fetch then creates another task to send email
  • everything stored in datastore and trashed once completed (no increase in size here)
  • retries on error (deadline exceptions, maintenance, unknonwn errors, etc)
  • emails are still sent if data if over limit (just no diff printed)

Enable billing

I recently evaluated that enabling the billing is not a bad option. Just by doing so, you are better off on those limitations, and will still not even be charged a dime. You can also set a small daily budget of $1 a day, just in case things go crazy there, so no big harm.

Upgrade Instance class

Another point to consider is upgrading the instance, if you can afford this. The initial 128MB RAM is not that much. Specially if you are commiting an iOS project or game for the first time, this can easily take more than 100MB of raw diff text, which needs to be parsed into HTML (becomes more than double of that), and all of this is done in memory... you see where this is going...

If you don't want to worry about it, but still have the billing enabled, consider leaving the BODY_MAX_SIZE with something like 1~2MB. The default is set to 340KB for the free tier, but the billing enabled accepts up to 86MB. Good but not good enough to handle all commmits...

However, the first commits, or commits with such amount of data, have little or no meaning for a visual check over email, so a reasonable limit like the above should suit best. Additionally, there is an issue with the capacity of the data (even zipped) stored in datastore. In case of such issues happen, an email without diff is still issued.

Bugs & Comments

Very welcome. Go to this project's Issues, or fork this code.

License

    Copyright 2013 Bruno Braga

    Licensed under the Apache License, Version 2.0 (the "License"); you may
    not use this file except in compliance with the License. You may obtain
    a copy of the License at

         http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
    License for the specific language governing permissions and limitations
    under the License.