S3BotoStorage - Filename not always unique when AWS_S3_FILE_OVERWRITE = False

Issue #116 new
Michael Godshall
created an issue

I'm experiencing inconsistent behavior with unique file names for S3BotoStorage. Sometimes it creates a new file name (like myusername_1.jpg), but it often overwrites old files with the same name. It is my understanding that by setting AWS_S3_FILE_OVERWRITE to False, new files will not overwrite old files even if they have the same name. For example, if we use the user's username as the name of the file, it would result in:


Files that are added in the future should increment like this:


As I mentioned above, sometimes it creates a new file in this pattern, but sometimes it overwrites an existing file. For example, earlier I had myusername.jpg, myusername_1.jpg, and myusername_2.jpg in an upload directory, with myusername_2.jpg set for the ImageField (the most recent). I would expect the next uploaded file to be named myusername_3.jpg, but myusername_1.jpg was overwritten instead.

S3BotoStorage relies on the inherited get_available_name() method from the core Django Storage to get a unique filename, so I'm wondering if S3 needs additional code for this to work consistently. If I should not rely on S3BotoStorage to ensure unique filenames, is there another approach you would recommend? Maybe I should prefix the filename with a timestamp?

Comments (2)

  1. Ian Lewis

    That's right, it should write myusername_3.jpg if _1 and _2 exist. Currently it makes a request to S3 for each file. Perhaps S3 is incorrectly reporting that the file doesn't exist?

    Unfortunately I'm not going to be terribly fast or reliable no matter what we do which is why this is an option to begin with, but if you are able to reproduce it and can offer more info or suggestions (or pull-requests!) then I'd be happy to look at them.

  2. Log in to comment