#62 Open

New FileCache storages: faster access for CDN files already accessed

Bitbucket cannot automatically merge this request.

The commits that make up this pull request have been removed.

Bitbucket cannot automatically merge this request due to conflicts.

Review the conflicts on the Overview tab. You can then either decline the request or merge it manually on your local system using the following commands:

hg update default
hg pull -r filecache_storage_proposal https://bitbucket.org/alanjds/django-storages
hg merge filecache_storage_proposal
hg commit -m 'Merged in alanjds/django-storages/filecache_storage_proposal (pull request #62)'
  1. Alan Justino da Silva

The new storage 'storages.backends.filecache.FileSystemCachedS3BotoStorage' makes faster thumbnail creation (e.g. for easy_thumbnail) or whatever operation performed by Django on files hosted at S3.

its heavily based on a extendable FileCacheMixin, so can be easily adapted for other remote storages.

On .read() operations, it transfers the file from slower master storage (S3 in this case) to faster cache storage (FileSystemCache in this case), so next uses needs no new transfer. Note that common usage have no .read() operations, and .url() keeps proxing to master storage. This cache only affects .read() operations, usually Django operations.

On .write(), cache storage version is cleaned, so making chance for master storage to do fancy changes/optimizations and transfer fresh files on next .read() call

Other default operations are just proxied to master storage.

For bonus, the implemented FileSystemCachedS3BotoStorage have a working .path() for S3 based files, allowing transparent integration with django_compressor and other tools that encapsulate calls to filesystem tools.

The only caveat is that, for now, it does not implement external invalidation, so a new file uploaded to master storage (S3) from outside of Django will NOT trigger cache storage invalidation. However, this is possible and planned to be implemented later.

Comments (0)