Issues

Issue #11 resolved

Windows Paths and S3 Keys

Anonymous created an issue

With S3 Storage - if running on windows then the backslash is used in the key name which is incorrect. I fixed by adding to the first line of S3.AWSAuthConnection._make_request

key = key.replace("\", "/")

Comments (13)

  1. Anonymous

    This issue tracker treated double backslash as a newline character, so my example is broken

  2. Anonymous

    I just got bitten by this bug also. The proposed solution seems simple and effective enough.

  3. Jason Christa

    Perhaps related to this issue, is paths that start with './folder/filename' don't have the '.' removed before uploading to S3 so there is a folder with the name '.'

  4. Jason Christa
    • changed status to open

    The '\' is not a problem with S3. '\' is a valid character in the url so S3 happily accepts it. The problem is since Django actually saves the paths in the db with '/' you have to take care to make sure S3 represents the file path the same way.

  5. Kyle MacFarlane

    This is a huge problem with S3. "folder\file.jpg" is a file with a backslash in its name while "folder/file.jpg" is the file "file.jpg" inside the folder "folder". This makes a huge difference when it comes to ACL settings.

    I understand that it's a problem with the S3 lib but that hasn't been touched for two years.

  6. Jason Christa

    Hey David,

    What is your stance on this? I still don't believe that this an S3 lib problem just because S3 doesn't claim support folders, they simply just allow file names with a '/' in them. Django's file storage converts between '/' and '\' depending on the platform, S3 and S3boto should also take care of the slashes.

  7. David Larlet repo owner

    Jason,

    I see 3 options (if you see another please let me know, I'm very busy those days and I haven't enough time to think about that):

    • the error comes from the S3 lib, I can fix it locally but the upstream developers should be contacted about this issue ;
    • the error comes from how Django handle "\" in file names and in this case it should be fixed in Django, I can fix that at the custom storage level but the place is in Django ;
    • the error comes from the storage and in this case I need a patch against it (I hadn't any Windows to test that).

    Thoughts?

  8. Jason Christa
    • I still think the S3 lib should not reject or coerce a valid url, 'my\folder\file' is a valid url.
    • It would be nice if Django "asked" the storage what separator it should use when providing file paths to save but it doesn't and that would still require a small change to the storage
    • I think the storage should be "smart" enough to determine we are trying to use folders inside of S3 and coerce any file paths it gets to be a valid S3 folder. E.g. 'my\folder\file' -> 'my/folder/file' and even './my/folder/file' -> 'my/folder/file'.
  9. Log in to comment