HTTPS SSH

BBCloner

About

BBCloner (Bitbucket Cloner) creates mirrors of your public and private Bitbucket Git repositories and wikis. It also synchronizes already existing mirrors. Initial mirror setup requires you manually enter your username/password. Subsequent synchronization of mirrors is done using Deployment Keys.

Features

  • Clone / mirror / backup public and private repositories and wikis.
  • No need to store your username and password to update clones.
  • Exclude repositories.
  • No need to run an SSH agent. Uses passwordless private Deployment Keys. (thus without write access to your repositories)

WARNING: Do NOT add a passwordless SSH key in your Account's SSH keys. Add them as Deployment keys to separate repositories. Deployment keys have only read-only access. Account keys have full write access, thus compromising your source code if your backup machine is compromised.

Requirements and Installation

If you're downloading an official tar.gz, zip or .deb release, bbcloner requires the following:

  • Python 2.6+
  • Git

The Debian/Ubuntu package automatically satisfies these dependencies.

Source install (.tar.gz)

If you're installing from source:

  1. Download the .tar.gz file from the download page.
  2. Unpack it:

    $ tar -vxzf bbcloner-*.tar.gz
    
  3. Install it:

    $ cd bbcloner-* && sudo ./install.sh
    

Debian / Ubuntu package

Installing the Debian/Ubuntu package:

  1. Download the .deb file from the download page.
  2. Install the package:

    $ sudo dpkg -i ./bbcloner-*.deb
    

From the Git repository

If you're cloning the repository directly:

  1. Get the repository

    $ git clone git@bitbucket.org:fboender/bbcloner.git
    
  2. Install python-requests

    pip install requests
    

Usage

Usage: ./bbcloner.py <path>

Bitbucket Cloner. Clone or update mirrors of Git bitbucket repos into <path>

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -k SSHKEY, --key=SSHKEY
                        Path to passwordless SSH deployment key.(default:
                        ~/.ssh/bbcloner_rsa)
  -u USERNAME, --user=USERNAME
                        Bitbucket username. If not specified, only updates
                        existing mirrors.If you don't specify --password, it
                        will prompt for a password.
  -l, --link LINK_TYPE  Use LINK_TYPE for cloning repositories. LINK_TYPE: git or http.
                        Default: git
  -p PASSWORD, --password=PASSWORD
                        Bitbucket password
  --skip=SKIP           Comma separated list of repo names to skip when
                        creating new mirrors. Existing mirrors with this name
                        will still be updated
  -n, --no-update       Do no update existing mirrors, only clone new ones
  -w, --no-wiki         Do no clone wikis
  -s, --silent          Only show errors
  -t, --tolerant        Only report update error if the repo failed last time
                        bbcloner was run

Example usage

  1. Generate a new passwordless private / public SSH key:

    $ ssh-keygen
    Generating public/private rsa key pair.
    Enter file in which to save the key: /home/fboender/.ssh/bbcloner_rsa<ENTER>
    Enter passphrase (empty for no passphrase):<ENTER>
    Enter same passphrase again: <ENTER>
    
  2. Add the public SSH key as a Deployment Key for each repository you wish to mirror.

  3. Initially, or whenever you've created new repositories in Bitbucket, manually run bbcloner with your username/password:

    $ mkdir /home/fboender/gitclones/
    $ bbcloner -n -u fboender /home/fboender/gitclones/
    Cloning new repositories
    Cloning project_a
    Cloning project_a wiki
    Cloning project_b
    
  4. Subsequently, to synchronize your mirrors:

    $ bbcloner /home/fboender/gitclones/
    Updating existing mirrors
    Updating /home/fboender/gitclones/project_a.git
    Updating /home/fboender/gitclones/project_a-wiki.git
    Updating /home/fboender/gitclones/project_b.git
    

    You can use the following cronjob to automatically synchronise your mirrors once every 6 hours:

    $ crontab -e
    0 */6 * * * bbcloner -s /home/fboender/git/
    

Using mirrored repositories

In case of downtime of Bitbucket, and you don't have a clone of a remote repository already, you can clone the mirror:

$ git clone /home/fboender/gitclones/project_a.git/
Cloning into project_a...
done.

Or, from a remote machine via SSH:

$ git clone ssh://example.com/home/fboender/gitclones/project_a.git
Cloning into 'project_a'...
X11 forwarding request failed on channel 0
remote: Counting objects: 634, done.
remote: Compressing objects: 100% (274/274), done.
remote: Total 634 (delta 215), reused 634 (delta 215)
Receiving objects: 100% (634/634), 87.36 KiB, done.
Resolving deltas: 100% (215/215), done.

If you're going to make commits in a checkout of a mirror, be sure to remove the mirror as an remote origin and add the Bitbucket remote origin before you push:

$ cd project_a
$ git remote rm origin
$ git remote add origin git@bitbucket.org:fboender/project_a.git
$ git push
remote: bb/acl: fboender is allowed. accepted payload.

Bugs

  • No support for Hg.
  • Bitbucket's REST API is currently broken, so you have to manually add Deployment Keys to each of your repositories.
  • No support for backups of Issues.

Patches are welcome.

bbcloner is licensed under the following (MIT) license:

Copyright (C) 2013-2015 Ferry Boender ferry.boender@gmail.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.