Overview

ec2-snapshot-filesystem

This is a wrapper script around Eric Hammond's ec2-consistent-snapshot, which can flush and READ LOCK a MySQL database and freeze XFS filesystems before taking a snapshot of an EBS block device.

In order to use snapshots for backup purposes, ec2-snapshot-filesystem (this tool) tries to simplify the process even more. Basically, all you need is to invoke (as root)

ec2-snapshot-filesystem {path-to-mount-point}

It will then

  • Figure out the EBS volume IDs for the device at the given mount point.
  • Support MD (software RAID) devices. If software RAID is used at the given mount point, snapshot all underlying EBS volumes simultaneously.
  • Use the right parameters to freeze the filesystem if it is XFS
  • Use the login information from /etc/mysql/debian.cnf to check if you have MySQL running and the databases are kept under the given mount point. If so, add the necessary parameters to lock and flush the MySQL server.
  • Set the description for the snapshot taken stating your instance-ID, mount point and time.
  • Clean up older snapshots to keep only snapshots according to a hard-coded policy (see code for details).

Installation and usage

  1. This is written in PHP as that's what I am most fluent in - so you need to have the PHP CLI (command line SAPI) installed. Under Debian/Ubuntu, that's in the php-cli package.
  2. To manage EBS snapshots, the EC2 command line tools need to be installed. Under Debian/Ubuntu, these are in the ec2-api-tools package.

Additionally, to provide ec2-consistent-snapshot as well as the EC2 command line tools with the necessary credentials and URL endpoints, the following environment variables probably need to be set. Refer to ec2-consistent-snapshot --help and the EC2 command line tools documentation for detailed setup instructions.

  • AWS_CREDENTIALS should point to the file containing both the Amazon AWS access key and secret access key on seprate lines and in that order. It is read by ec2-consistent-snapshot.
  • EC2_URL is the AWS endpoint, for example https://ec2.eu-west-1.amazonaws.com. Like the following two variables it is used by the EC2 command line tools.
  • EC2_CERT is the path to the AWS X.509 certificate file used for authentication
  • EC2_PRIVATE_KEY is the path to the AWS private key

If you got all those right, the basic invocation is

ec2-snapshot-filesystem {path-to-mount-point}

Any additional parameters will be passed on to ec2-consistent-snapshot, so you can add -n for no-op mode and --debug to turn ec2-consistent-snapshot's debugging on.

Output

ec2-snapshot-filesystem will emit some lines like the following: Purging old snapshots...

+ Sun, 07 Aug 2011 04:37:11 +0200
+ Sun, 04 Sep 2011 04:37:11 +0200
+ Sun, 11 Sep 2011 04:37:13 +0200
+ Sun, 18 Sep 2011 04:37:12 +0200
+ Sun, 25 Sep 2011 04:37:13 +0200
- Sat, 01 Oct 2011 04:37:13 +0200
+ Sun, 02 Oct 2011 04:37:10 +0200
+ Mon, 03 Oct 2011 04:37:12 +0200
+ Tue, 04 Oct 2011 04:37:14 +0200
+ Wed, 05 Oct 2011 04:37:12 +0200
+ Thu, 06 Oct 2011 04:37:12 +0200
+ Fri, 07 Oct 2011 04:37:13 +0200

Lines starting with a "+" indicate snapshots taken on this mount point that were kept according to the backup policy. "-" indicates a snapshot that has just been deleted because it is no longer required.