html_error_template doesn't handle non-ascii characters in source code
Max Ischenko reported this here:
I've attached a patch to convert the source code to unicode via the source code's magic encoding comment (like Myghty does -- in fact I pretty much ripped all of dairiki's code for this). In the rare case that the magic encoding comment doesn't describe the non-ascii characters correctly, the 'replace' error handler is used to add garbage in its place
But before we commit it, I think I would like to see an addition to the Template class.
Pylons currently does this in its exception handler to render the exception:
With this patch, and Max's example, Pylons now causes a UnicodeEncodeError here:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 4-9: ordinal not in range(128)
What's needed for Pylons to render the html_error_template correctly (including the non-ascii characters showing up in the browser), it now has to do this:
return mako.exceptions.html_error_template().render_unicode().encode(sys.getdefaultencoding(), 'htmlentityreplace')[610:-16]
Mike I may have asked you this before, but is there a reason why mako Templates don't have an 'encoding_errors' parameter to go along with 'output_encoding'?
If it did, we could have html_error_template set encoding_errors='htmlentitiyreplace'. Then anyone needing an html_error_template wouldn't have to be concerned with unicode at all -- they could simply render() (like Pylons is doing now) and always get back a sys.getdefaultencoding() encoded string with html entitys for non-defaultencoding characters. That is what Myghty did:
I remember arguing this implementation a little with dairiki when he did this, but it seemed to work out pretty well