Wiki

Clone wiki

sftpgateway-public / Remove empty folders after download sync

Private and shared download folders perform a one-way sync from S3 to the local Linux file system.

The problem is that this sync does not clean up empty folders. The AWS SDK by default does not delete empty folders when it performs sync operations. This can lead to a clutter of empty folders which makes for a poor user experience.

One workaround is to run a command that deletes all empty folders and subfolders:

sudo su
cd /home/robtest/home/robtest/downloads/
find . -type d -empty -delete

You can turn this into a script. Create a file called /root/cleanup-folders.sh, and add the following contents:

#!/bin/bash

find $1 -type d -empty -delete

Don't forget to make this script executable:

chmod +x /root/cleanup-folders.sh

This is how you use the script:

/root/cleanup-folders.sh /home/robtest/home/robtest/downloads/

The next step is to schedule this script:

sudo su
crontab -e

In the crontab, you should already see an existing entry for s3sync:

*/5 * * * * /usr/local/bin/s3sync > /dev/null 2>&1

Append an additional entry for your folder cleanup script:

*/5 * * * * /usr/local/bin/s3sync > /dev/null 2>&1
* * * * * /root/cleanup-folders.sh /home/robtest/home/robtest/downloads/ > /dev/null 2>&1

This runs the folder cleanup script every minute, and discards any command line output or errors.

Note: Editing crontab uses vi. Here are a few vi commands for reference:

  • $: Go to the end of the current line
  • i: Switch to insert mode
  • escape: Switch to command mode
  • :wq: In command mode, this writes and quits.

Updated