Unicode in warning message results in unicode error being reported

Issue #165 new
Jonathan Pyle
created an issue

The warning message about duplicate keys includes some of the YAML source in the error message. If the YAML source contains Unicode, however, the printing of the warning message itself produces a unicode error.

One could get around this by disabling the warning. However, when this happened to me, it was hard to figure out that "duplicate keys" was even the problem, since the only error I was seeing was "ordinal not in range."

Maybe when warnings and errors are printed along with the source context, there could be a try/except on printing the source context, and if there is an exception, the warning/error could be printed in a way that does not include the source context.

>>> import ruamel.yaml
>>> ruamel.yaml.safe_load('type: Domestic\ntypetwo: International')
{'typetwo': 'International', 'type': 'Domestic'}
>>> ruamel.yaml.safe_load('type: Domestic\ntype: International')
ructor.py:262: DuplicateKeyFutureWarning: while constructing a mapping
  in "<byte string>", line 1, column 1:
    type: Domestic
    ^ (line: 1)
found duplicate key "type" with value "International" (original value: "Domestic")
  in "<byte string>", line 2, column 1:
    type: International
    ^ (line: 2)

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

Duplicate keys will become and error in future releases, and are errors
by default when using the new API.

  warnings.warn(DuplicateKeyFutureWarning(*args))
>>> ruamel.yaml.safe_load('type: Doméstica\ntypetwo: International')
{'typetwo': 'International', 'type': u'Dom\xe9stica'}
>>> ruamel.yaml.safe_load('type: Doméstica\ntype: International')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/share/docassemble/local/local/lib/python2.7/site-packages/ruamel/yaml/main.py", line 664, in safe_load
    return load(stream, SafeLoader, version)
  File "/usr/share/docassemble/local/local/lib/python2.7/site-packages/ruamel/yaml/main.py", line 635, in load
    return loader._constructor.get_single_data()  # type: ignore
  File "/usr/share/docassemble/local/local/lib/python2.7/site-packages/ruamel/yaml/constructor.py", line 104, in get_single_data
    return self.construct_document(node)
  File "/usr/share/docassemble/local/local/lib/python2.7/site-packages/ruamel/yaml/constructor.py", line 114, in construct_document
    for dummy in generator:
  File "/usr/share/docassemble/local/local/lib/python2.7/site-packages/ruamel/yaml/constructor.py", line 611, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/usr/share/docassemble/local/local/lib/python2.7/site-packages/ruamel/yaml/constructor.py", line 356, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/usr/share/docassemble/local/local/lib/python2.7/site-packages/ruamel/yaml/constructor.py", line 238, in construct_mapping
    self.check_mapping_key(node, key_node, mapping, key, value)
  File "/usr/share/docassemble/local/local/lib/python2.7/site-packages/ruamel/yaml/constructor.py", line 251, in check_mapping_key
    key, value, mapping.get(key)), key_node.start_mark,
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 3: ordinal not in range(128)

Comments (0)

  1. Log in to comment