SymlinkOrCopyStorage: new custom storage system

Issue #6 resolved
Wim Leers created an issue

This patch has been created as part of my bachelor thesis [1] – as part of the daemon I wrote. I'm using django-storages to send files to FTP, Amazon S3 and Amazon CloudFront.

I needed to efficiently create a mirror of a directory tree (so that "origin pull" CDNs can automatically pull files). The trick was that some files could be modified, and some could be identical to the original. Of course it doesn't make sense to store the exact same data twice on the file system. So I created SymlinkOrCopyStorage.

SymlinkOrCopyStorage allows you to symlink a file when it's identical to the original file and to copy the file if it's modified. Of course, it's impossible to know if a file is modified just by looking at the file, without knowing what the original file was. That's what the symlinkWithin parameter is for. It accepts one or more paths (if multiple, they should be concatenated using a colon (:)). Files that will be saved using SymlinkOrCopyStorage are then checked on their location: if they are within one of the symlinkWithin directories, they will be symlinked, otherwise they will be copied.

The rationale is that unmodified files will exist in their original location, e.g. /htdocs/ and modified files will be stored in a temporary directory, e.g. /tmp/image.jpg.

I'm looking forward to your comments, questions and criticism :)


Comments (4)

  1. David Larlet repo owner
    • changed status to open

    I'm not sure it'll be useful to a lot of people but I'll add it, we'll see :)

    By the way, is there a reason to do not use super() instead of FileSystemStorage "direct" calls?

  2. Wim Leers reporter

    I'm new to Python since about a month. I read Dive Into Python.

    I've been recommended by a fellow student who's been programming Python for more than a year that not using super() but using the parent class name is the appropriate style. I guess that's not true then?

    It probably won't be useful to a large audience, but it's not exactly much code either, so I don't think it'll hurt :)

  3. David Larlet repo owner

    I added he storage, let me know if it still works (I made a few modifications) and I'll close the issue.

  4. Log in to comment