We have developed the following process for recovering SFTP Gateway users and configurations. This will be useful in the following cases:

• A region outage requires you to do a cold failover to another region
• You need to migrate or replicate to another region
• An SFTP Gateway instance is accidentally deleted
• You are upgrading from a single SFTP Gateway instance to a high availability stack

Note: The backup and recovery process captures the credentials for both password and SSH key based authentication.

Make sure you test this process thoroughly in your own environment.

Disclaimer: This backup and recovery strategy is intended for standard installations of SFTP Gateway. If you have customized your SFTP Gateway installation(s), please contact us at support@thorntech.com to see if this solution will work for you.

Backup

Creating a backup

The SFTP Gateway backup script is a Python script that you install on your SFTP Gateway instance. The script backs up the following data to a single flat YAML file (and later gets compressed as a tar.gz file).

• user properties
• user passwords/keys
• global SFTP Gateway properties

This backup artifact can be imported into a new SFTP Gateway stack to restore settings and configuration.

The sftpgw-backup.py script will create a backup file named sftpgw-backup-YYYY-MM-DD-HH-MM-SS.tar.gz. By default, this file is saved in the/home/ec2-user/backups/ directory (which gets created if it doesn't already exist).

To run the backup script:

1. SSH to the server as the ec2-user (https://help.thorntech.com/help/log-into-the-ec2-instance)
2. Download the backup script

cd /home/ec2-user/
wget https://s3.amazonaws.com/thorntech-public-documents/sftpgateway/backup-and-recovery/sftpgw-backup.py

1. Run the backup script

sudo python sftpgw-backup.py

This will create a backup file sftpgw-YYYY-MM-DD-HH-MM-SS.tar.gz. This will later be used to recover SFTP Gateway users and configurations.

(Optional) Use the --destination-dir parameter to save the backup artifact to a specific location.

sudo python sftpgw-backup.py --destination-dir /path/to/backup/location/

This parameter will create the directory if it doesn't exist. The script will fail if the destination is a file.

Next steps

Once a backup artifact has been created, you can copy it to S3 or some other external location. This can be done using a "backup" SFTP user and saving the file to that user's uploads directory.

To copy the backup artifact using SFTP Gateway:

1. Create a "backup" user with the web interface, CLI, or API
2. You can point this user's uploads to a specific S3 bucket
3. Run the backup script with the backup user's /home/bacup-user/home/backup-user/uploads directory as the destination

sudo python sftpgw-backup.py --destination-dir /home/backup/home/backup/uploads

This will upload the backup file to S3 as soon as it is created. Storing the backup artifact on S3 will improve its availability should something happen to the SFTP Gateway instance.

Alternatively, you can copy the backup artifact to an external location manually, or in an automated fashion.

Schedule backup

A scheduled backup can help improve business continuity since your user settings will be up to date as far as the most recent backup.

To schedule a backup, you can create a cron job to run the backup script.

Here is an example for running the backup at 11:00 PM every Friday:

1. Open the root crontab for editing. This will open the crontab in vim¹

sudo crontab -e

1. Enter the following as a new line

0 23 * * Fri PATH=$PATH:/usr/local/bin; python /home/ec2-user/sftpgw-backup.py --destination-dir /home/ec2-user/backups >> /var/log/sftpgw/backup.log 2>&1

This is a breakdown of the syntax:

• 0 23 * * Fri - Time schedule that runs script at 23:00 server time every Friday.
• PATH=$PATH:/usr/local/bin; - Adds /usr/local/bin to the working path for this cronjob.
• python /home/ec2-user/sftpgw-backup.py --destination-dir /home/ec2-user/backups - Runs the script and writes the backup files to /home/ec2-user/backups.
• >> /var/log/sftpgw/backup.log 2>&1 - Directs all output to a specific log.

Here is a good resource for building cronjob schedules - https://crontab.guru/

Recovery

To recover SFTP Gateway users and configurations:

1. Connect to the server as the ec2-user over SSH (https://help.thorntech.com/help/log-into-the-ec2-instance)
2. Download the recovery script:

cd /home/ec2-user/
wget https://s3.amazonaws.com/thorntech-public-documents/sftpgateway/backup-and-recovery/sftpgw-recovery.py

1. Copy the sftpgw-backup-YYYY-MM-DD-HH-MM-SS.yml.tar.gz file to the server. For example, if the backup file is stored in S3, you can use the aws s3 cp command:

aws s3 cp s3://bucketname/backups/sftpgw-backup-YYYY-MM-DD-HH-MM-SS.tar.gz /home/ec2-user/backup_files/

1. Run the python recovery script

sudo python sftpgw-recovery.py sftpgw-YYYY-MM-DD-HH-MM-SS.tar.gz

Your new instance will be in the same state as the original instance, including your users and SFTP Gateway configuration.

Foot Notes