Commits

Anonymous committed 659ee15

[svn] Make RawTokenFormatter/Lexer use unicode escaped strings.

  • Participants
  • Parent commits a16ef78
  • Branches trunk

Comments (0)

Files changed (2)

File pygments/formatters/other.py

         lasttype = None
         lastval = u''
         for ttype, value in tokensource:
-            value = value.encode(self.encoding)
+            value = repr(value)
             if ttype is lasttype:
                 lastval += value
             else:

File pygments/lexers/special.py

                     for ttype_ in ttypes:
                         ttype = getattr(ttype, ttype_)
                     _ttype_cache[ttypestr] = ttype
-                val = val[1:-2].decode('string-escape')
+                val = val[2:-2].decode('unicode-escape')
             yield length, ttype, val
             length += len(val)