#62 Open
Repository
alanjds alanjds
Branch
filecache_storage_proposal
Repository
david david
Branch
default

New FileCache storages: faster access for CDN files already accessed

Bitbucket cannot automatically merge this request due to conflicts.

Review the conflicts on the Overview tab. You can then either decline the request or merge it manually on your local system using the following commands:

hg update default
hg pull -r filecache_storage_proposal https://bitbucket.org/alanjds/django-storages
hg merge filecache_storage_proposal
hg commit -m 'Merged in alanjds/django-storages/filecache_storage_proposal (pull request #62)'
Author
  1. Alan Justino avatarAlan Justino
Reviewers
Description

The new storage 'storages.backends.filecache.FileSystemCachedS3BotoStorage' makes faster thumbnail creation (e.g. for easy_thumbnail) or whatever operation performed by Django on files hosted at S3.

its heavily based on a extendable FileCacheMixin, so can be easily adapted for other remote storages.

On .read() operations, it transfers the file from slower master storage (S3 in this case) to faster cache storage (FileSystemCache in this case), so next uses needs no new transfer. Note that common usage have no .read() operations, and .url() keeps proxing to master storage. This cache only affects .read() operations, usually Django operations.

On .write(), cache storage version is cleaned, so making chance for master storage to do fancy changes/optimizations and transfer fresh files on next .read() call

Other default operations are just proxied to master storage.

For bonus, the implemented FileSystemCachedS3BotoStorage have a working .path() for S3 based files, allowing transparent integration with django_compressor and other tools that encapsulate calls to filesystem tools.

The only caveat is that, for now, it does not implement external invalidation, so a new file uploaded to master storage (S3) from outside of Django will NOT trigger cache storage invalidation. However, this is possible and planned to be implemented later.

Comments (0)

Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.