UnicodeEncodeError

Issue #642 resolved
Nuno Khan
created an issue

I am getting a 'ascii' codec can't encode characters in position 6-7: ordinal not in range(128) when i try to save with version 2.4.0-a1 and 2.4.0-b1

Everything works fine with version 2.3.5 so the issue is probably not with my code. i am using Python 2.7

Traceback:
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/options.py" in wrapper
  618.                 return self.admin_site.admin_view(view)(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapped_view
  110.                     response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/views/decorators/cache.py" in _wrapped_view_func
  57.         response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/sites.py" in inner
  233.             return view(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapper
  34.             return bound_func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapped_view
  110.                     response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in bound_func
  30.                 return func.__get__(self, type(self))(*args2, **kwargs2)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/options.py" in changelist_view
  1576.                 response = self.response_action(request, queryset=cl.get_queryset(request))
File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/options.py" in response_action
  1335.             response = func(self, request, queryset)
File "/vagrant/stations/admin.py" in export_rating_xls
  379.     wb.save(response)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/workbook/workbook.py" in save
  281.             save_workbook(self, filename)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/writer/excel.py" in save_workbook
  248.     writer.save(filename, as_template=as_template)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/writer/excel.py" in save
  231.         self.write_data(archive, as_template=as_template)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/writer/excel.py" in write_data
  78.         self._write_worksheets(archive)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/writer/excel.py" in _write_worksheets
  167.             xml = sheet._write(self.workbook.shared_strings)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/worksheet/worksheet.py" in _write
  839.         return write_worksheet(self, shared_strings)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/writer/worksheet.py" in write_worksheet
  161.                 xf.write(ws.HeaderFooter.to_tree())
File "/usr/local/lib/python2.7/dist-packages/openpyxl/descriptors/serialisable.py" in to_tree
  138.                     node = obj.to_tree(child_tag)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/worksheet/header_footer.py" in to_tree
  182.         el.text = str(self)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/worksheet/header_footer.py" in __str__
  166.                 txt.append("&{0}{1}".format(key, str(part)))

Exception Type: UnicodeEncodeError at /admin/stations/station/
Exception Value: 'ascii' codec can't encode characters in position 6-7: ordinal not in range(128)

Comments (27)

  1. CharlieC

    Yeah, the HeaderFooter code was completely rewritten for 2.4 (and is more extensive). If you're already running with 2.4, can you work with a checkout?

  2. Nuno Khan reporter

    @CharlieC i am still having the same problem even after your fix

    File "/usr/local/lib/python2.7/dist-packages/openpyxl/descriptors/serialisable.py" in to_tree
      138.                     node = obj.to_tree(child_tag)
    File "/usr/local/lib/python2.7/dist-packages/openpyxl/worksheet/header_footer.py" in to_tree
      199.         el.text = str(self)
    File "/usr/local/lib/python2.7/dist-packages/openpyxl/worksheet/header_footer.py" in __str__
      182.                 txt.append(u"&{0}{1}".format(key, str(part)))
    
    Exception Type: UnicodeEncodeError at /admin/stations/station/
    Exception Value: 'ascii' codec can't encode characters in position 6-7: ordinal not in range(128)
    
  3. Nuno Khan reporter

    @CharlieC i am now getting the same error someplace else:

    Traceback:
    File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response
      132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
    File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/options.py" in wrapper
      618.                 return self.admin_site.admin_view(view)(*args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapped_view
      110.                     response = view_func(request, *args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/django/views/decorators/cache.py" in _wrapped_view_func
      57.         response = view_func(request, *args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/sites.py" in inner
      233.             return view(request, *args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapper
      34.             return bound_func(*args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapped_view
      110.                     response = view_func(request, *args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in bound_func
      30.                 return func.__get__(self, type(self))(*args2, **kwargs2)
    File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/options.py" in changelist_view
      1576.                 response = self.response_action(request, queryset=cl.get_queryset(request))
    File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/options.py" in response_action
      1335.             response = func(self, request, queryset)
    File "/vagrant/stations/admin.py" in export_rating_xls
      381.     wb.save(response)
    File "/usr/local/lib/python2.7/dist-packages/openpyxl/workbook/workbook.py" in save
      317.             save_workbook(self, filename)
    File "/usr/local/lib/python2.7/dist-packages/openpyxl/writer/excel.py" in save_workbook
      268.     writer.save(filename)
    File "/usr/local/lib/python2.7/dist-packages/openpyxl/writer/excel.py" in save
      250.         self.write_data()
    File "/usr/local/lib/python2.7/dist-packages/openpyxl/writer/excel.py" in write_data
      93.         archive.writestr(ARC_WORKBOOK, write_workbook(self.workbook))
    File "/usr/local/lib/python2.7/dist-packages/openpyxl/writer/workbook.py" in write_workbook
      133.                                   for r in sheet.print_area])
    
    Exception Type: UnicodeEncodeError at /admin/stations/station/
    Exception Value: 'ascii' codec can't encode character u'\xe3' in position 12: ordinal not in range(128)
    
  4. CharlieC

    Thanks. And here's me thinking you're going to submit a PR. I think I'm going to stop adding features and cleaning up code… ;-)

    Where's the file from? Excel doesn't like it.

  5. CharlieC

    Well, it opens fine in 2.4. You're seeing problems saving with 2.7 because of the fudge that Python 2 does with strings and unicode. This second part relates to the sheet names associated with the filters. These are lost in 2.3 but preserved in 2.4

    Interestingly 2.3 seems to preserve more of the formatting in the headers and footers than 2.4. Assuming the unicode is resolved I would consider this a bug but not one that would stop a release of 2.4.

  6. Nuno Khan reporter

    @CharlieC i was able to solve the issue and successfully import the excel by changing writer/workbook.py:

      from openpyxl.compat import unicode
           ....
            if auto_filter:
                name = DefinedName(name='_FilterDatabase', localSheetId=idx, hidden=True)
                name.value = "{0}!{1}".format(quote_sheetname(sheet.title).encode('utf-8'),
                                              absolute_coordinate(auto_filter)
                                              )
                name.value = unicode(name.value.decode('utf-8'))
                defined_names.append(name)
    
       ....
    
            if sheet.print_area:
                name = DefinedName(name="Print_Area", localSheetId=idx)
                name.value = ",".join(["{0}!{1}".format(quote_sheetname(sheet.title).encode('utf-8'), r) for r in sheet.print_area])
                name.value = unicode(name.value.decode('utf-8'))
    

    If you know of a better way let me know..

  7. CharlieC

    This is not Python 3 safe. But I've already fixed this locally.

    You might want to take a closer look at what happens with the content of the headers and footers. There are other problems that mean I don't think you can use openpyxl 2.4 with your files at the moment.

  8. Nuno Khan reporter

    @CharlieC oh ok i will get your latest commit then.

    My original template (i uploaded here) might have been corrupted (i saved it using libreoffice) i re-saved it using Office, nevertheless my headers seem to be ok (for what i need at least) when i add info to my template and save it using openpyxl (with both the corrupted one and the new one)

  9. CharlieC

    Excel really doesn't like the non-integer font size embedded in some text somewhere (openpyxl strips this formatting and is unlikely ever to support it). I don't blame the LO / OO guys but sometimes some of their OOXML support isn't brilliant and can't be relied upon. Although it tends to fail less often than Excel does.

    <x:si xmlns:x="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
      <x:r>
        <x:rPr>
          <x:b val="true" />
          <x:sz val="9" />
          <x:color rgb="FFDB002A" />
          <x:rFont val="Arial" />
          <x:family val="2" />
          <x:charset val="1" />
        </x:rPr>
        <x:t>³</x:t>
      </x:r>
      <x:r>
        <x:rPr>
          <x:b val="true" />
          <x:sz val="8.55" />
          <x:color rgb="FFDB002A" />
          <x:rFont val="Arial" />
          <x:family val="2" />
          <x:charset val="1" />
        </x:rPr>
        <x:t>
        </x:t>
      </x:r>
      <x:r>
        <x:rPr>
          <x:b val="true" />
          <x:sz val="9" />
          <x:color rgb="FFDB002A" />
          <x:rFont val="Arial" />
          <x:family val="2" />
          <x:charset val="1" />
        </x:rPr>
        <x:t>85%</x:t>
      </x:r>
    </x:si>
    

    I'm still seeing problems locally with some of the definedNames which probably relates to the obscure escaping rules.

    The header footer stuff is now actually 1:1 whereas 2.3 seems to include additional formatting for the footer for the first worksheet.

  10. Log in to comment