S3boto backend: ValueError: time data 'Thu, 07 Mar 2013 13:02:16 GMT' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'

Issue #156 resolved
Kostiantyn Rybnikov created an issue


Recently our "python manage.py collectstatic" started to fail with this error: ValueError: time data 'Thu, 07 Mar 2013 13:02:16 GMT' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'

Seems that amazon is responding with a date in a format of RFC1123, while s3boto backend is calling boto/utils.py:parse_ts, which due to which it tries to parse date in a format ISO8601.

For me it looks like a bug in django-storages boto backend (it should do something like RFC1123 = '%a, %d %b %Y %H:%M:%S %Z'; time.mktime(time.strptime(iso8601_time, RFC1123))), but it only started happening recently, so I'm not sure.


Comments (10)

  1. Kostiantyn Rybnikov reporter
    Traceback (most recent call last):
      File "./manage.py", line 14, in <module>
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
        self.execute(*args, **options.__dict__)
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
        output = self.handle(*args, **options)
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/django/core/management/base.py", line 351, in handle
        return self.handle_noargs(**options)
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 89, in handle_noargs
        self.copy_file(path, prefixed_path, storage, **options)
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 184, in copy_file
        if not self.delete_file(path, prefixed_path, source_storage, **options):
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 118, in delete_file
        target_last_modified = self.storage.modified_time(prefixed_path)
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/storages/backends/s3boto.py", line 441, in modified_time
        return parse_ts(entry.last_modified)
      File "/home/shp/.virtualenvs/shwp/local/lib/python2.7/site-packages/boto/utils.py", line 395, in parse_ts
        dt = datetime.datetime.strptime(ts, ISO8601_MS)
      File "/usr/lib/python2.7/_strptime.py", line 325, in _strptime
        (data_string, format))
    ValueError: time data 'Wed, 13 Mar 2013 12:45:49 GMT' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'
  2. Ian Lewis

    Yah, we've had similar problems before. This happens mostly because the date format that is returned from the S3 API isn't very strict. Thanks for posting the issue.

  3. Kostiantyn Rybnikov reporter

    @richleland I think it was (possibly) brought by this commit, not fixed. To fix it you'll need to write a bit "upgraded" version of parse_ts, I've provided all needed details in original post (e.g. to add attempt to parse RFC1123).

  4. Rich Leland

    Hmm, I just released django-storages 1.1.7 yesterday, which included the commit. Are you running 1.1.7 or 1.1.6? If you're not running 1.1.7 would you be willing to upgrade and see if it resolves the issue?

  5. Filip Wasilewski

    @richleland The S3 API seems to return more date formats than the boto's parsing function is able to handle. I have been running 1.1.6 backend without problems and only observed this issue after upgrading to 1.1.7.

  6. Log in to comment