Ensure babel i18n extactor works properly with non-ascii input
Roman Imankulov
Branch: imankulov/mako:fix-unicode-input
imankulov/mako:fix-unicode-input
Branch: zzzeek/mako:master
zzzeek/mako:master
Merged
#21 · Created  · Last updated
Description
If mako templates contain something like "_('Köln')", babel extractor converts it to pure ASCII so that resulting .po file would contain "K\xf6ln". Not all translation tools and translations are ready for such kind of escape sequences.
Babel allows message ids to be non-ascii, the plugin just has to return Unicode objects instead of ASCII strings (and that's exactly how Babel built-in Python and JavaScript extractors work).
This fix ensures mako extractor doesn't excape non-ascii symbols, works well both for Unicode and non-unicode input (there is a test for cp1251 encoding), and also provides a workaround for babel charset detector python-babel/babel#274.
If mako templates contain something like "_('Köln')", babel extractor converts it to pure ASCII so that resulting .po file would contain "K\xf6ln". Not all translation tools and translations are ready for such kind of escape sequences.
Babel allows message ids to be non-ascii, the plugin just has to return Unicode objects instead of ASCII strings (and that's exactly how Babel built-in Python and JavaScript extractors work).
This fix ensures mako extractor doesn't excape non-ascii symbols, works well both for Unicode and non-unicode input (there is a test for cp1251 encoding), and also provides a workaround for babel charset detector python-babel/babel#274.