_entries cache for S3boto backend is regenerated many times

Issue #193 new
Javier Rojas created an issue

we have a site that uses django-storages with S3 and the S3 boto backend.

whenever we use AWS_PRELOAD_METADATA = True , there are many requests that take a long time (3+ minutes) because (to the best of my knowledge) the cache in S3BotoStorage._entries is regenerated.

This cache gets regenerated with a frequency too large, for the site to have a decent performance. It even happens sometimes with consecutive requests.

What determines the update frequency for this cache?

I've noticed that this behavior dissappears when I set AWS_PRELOAD_METADATA = False

What are the consequences of doing that?

Comments (1)

  1. Ian Lewis

    _entries is stored in memory so if you are using the Django runserver with a large S3 bucket then this could regenerate often.

    Under normal circumstances (i.e. using a normal production server) _entries should only be generated once.

  2. Log in to comment