Issues

Issue #6 resolved

SymlinkOrCopyStorage: new custom storage system

Wim Leers
created an issue

This patch has been created as part of my bachelor thesis [1] – as part of the daemon I wrote. I'm using django-storages to send files to FTP, Amazon S3 and Amazon CloudFront.

I needed to efficiently create a mirror of a directory tree (so that "origin pull" CDNs can automatically pull files). The trick was that some files could be modified, and some could be identical to the original. Of course it doesn't make sense to store the exact same data twice on the file system. So I created SymlinkOrCopyStorage.

SymlinkOrCopyStorage allows you to symlink a file when it's identical to the original file and to copy the file if it's modified. Of course, it's impossible to know if a file is modified just by looking at the file, without knowing what the original file was. That's what the symlinkWithin parameter is for. It accepts one or more paths (if multiple, they should be concatenated using a colon (:)). Files that will be saved using SymlinkOrCopyStorage are then checked on their location: if they are within one of the symlinkWithin directories, they will be symlinked, otherwise they will be copied.

The rationale is that unmodified files will exist in their original location, e.g. /htdocs/example.com/image.jpg and modified files will be stored in a temporary directory, e.g. /tmp/image.jpg.

I'm looking forward to your comments, questions and criticism :)

[1] http://wimleers.com/tags/bachelor-thesis

Comments (4)

  1. David Larlet repo owner
    • changed status to open

    I'm not sure it'll be useful to a lot of people but I'll add it, we'll see :)

    By the way, is there a reason to do not use super() instead of FileSystemStorage "direct" calls?

  2. Wim Leers reporter

    I'm new to Python since about a month. I read Dive Into Python.

    I've been recommended by a fellow student who's been programming Python for more than a year that not using super() but using the parent class name is the appropriate style. I guess that's not true then?

    It probably won't be useful to a large audience, but it's not exactly much code either, so I don't think it'll hurt :)

  3. Log in to comment