Rainbow: Characters getting deleted in ICML-to-XLIFF pipeline with Chinese source text

[Martn Wunderl]

When running the attached ICML file (UTF8 encoded) through a simple pipeline to create an XLIFF 2.0 file, some full-width characters are getting deleted.

In particular the following characters are missing from the XLIFF output:

0xFF08 65288 FULLWIDTH LEFT PARENTHESIS

0xFF09 65289 FULLWIDTH RIGHT PARENTHESIS

0xFF0C 65292 FULLWIDTH COMMA

0xFF1F 65311 FULLWIDTH QUESTION MARK

I'll try and look into it more closely. (assigning the issue to myself doesn't work for some reason)

Attached: ICML, Pipeline def, and generated XLIFF.

[Jim Hargrave (OLD)]

Should be fixed with ICML refactor.