Problems when using symlinked directories

[Todd Steinwart]

Hi! Overall great product and we anticipate a rollout to our customers early next year.

Here's what I'm struggling with: While your product addresses at-rest encryption on S3 a number of different ways, in order for us to meet security requirements of our customers we need at-rest encryption on the gateway instance as well - specifically the downloads/ directory.

My first thought was Encrypted EBS volumes but symlinking has proved problematic.

I tried to symlink the user's entire sftp directory structure:

If this is my encrypted mount:

/mnt/encrypted/

I create the user's target (all commands run as root):

mkdir todd 
cd todd
mkdir downloads
mkdir uploads
mkdir local
cd ..
chown -R todd:todd todd

now back to the root volume:

cd /home/todd/home
rm -rf todd
ln -s /mnt/encrypted/todd todd

Things look good and partially work: If I copy a file to the S3 downloads location it shows up in sftp after the next sync. The put command doesn't fare so well. Basically nothing happens (silent fail) from the sftp user's perspective (and no files get copied to S3). I tried chmod 777 uploads on the mount to no avail.

I thought a more targeted symlink might be better: only the downloads directory, reverting the rest of the tree to the root volume. Uploads went back to normal, but when I tried to cd into the downloads directory I got this error:

sftp> cd downloads
Couldn't canonicalize: No such file or directory

even though ls lists downloads/ among the available assets.

The canonicalize error I believe means it can't follow the symlink, as if it were not expecting a symlink in that location.

It's possible, though unlikely, I have a problem with the symlinks or their underlying permissions.

Have you encountered anything like this or thought through how to address at-rest crypto for the gateway? We can introduce code into the pipeline to programmatically encrypt the files we pass back to our clients but always prefer to use AWS tools where possible.

I'm sure I haven't explained things as clearly as I'd like to have so please ask any questions that will help clarify.

Thanks! Todd

[Robert Chen]

Hi Todd,

Thanks for using SFTP Gateway, and for providing those command line steps.

Permissions could get tricky on the /home/todd/home/todd folder, because this ChrootDirectory needs to be owned and exclusively writable by root (otherwise the user cannot connect via SFTP). Turning this directory into a symlink complicates things.

Mounting each user's individual downloads directory could be problematic. You would have to repeat this mount process for each new user. Also, this doesn't cover files that end up in the local and shared directories.

The simplest approach might be to just mount the entire /home as an encrypted EBS mount, which is something that we've considered exploring in the past.

I just tested this and documented the steps in this wiki page: https://bitbucket.org/thorntechnologies/sftpgateway-public/wiki/Encrypting%20local%20server%20data%20at%20rest

Give it a try and let me know how it goes.

Thanks! Robert

[Todd Steinwart]

Hi Robert!

Thanks for the thoughtful reply. I too had considered what you propose but wasn't sure whether I would have any more success than I had with my other attempts. The reasoning behind the permissions certainly makes sense and I will try out what you suggest.

That process is actually not new to me - I did it a while back for reasons that escape me now (not encryption-related) though we were ssh-ing into servers to write code.

Your point about the shared/ and local/ subdirs is well-taken but for our use case we only require uploads/ and downloads/. It's great your solution covers them as well because who know what the future holds.

Cheers! Todd

[Robert Chen]

Let me know how it goes. I'm open to suggestions on improving this process, so your any insights from your past experience are welcome. For example, someone already suggested to me to tar and zip the entire home directory in case there are other existing users that need migrating, and somehow the tar would better handle special cases like hard-links (which the shared directory uses).

[Todd Steinwart]

Will do. It'll be the weekend before I can circle back to this but I definitely will.