obj-c lexer fails on certain files

Issue #693 resolved
Aman Gupta
created an issue


import pygments, pygments.lexers, pygments.formatters

pygments.highlight("-initInManagedObjectContext:(NSManagedObjectContext)aContext withUser:(User)aUser\n", pygments.lexers.get_lexer_by_name("objective-c"), pygments.formatters.get_formatter_by_name("html")) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "pygments/pygments/init.py", line 86, in highlight return format(lex(code, lexer), formatter, outfile) File "pygments/pygments/init.py", line 66, in format formatter.format(tokens, realoutfile) File "pygments/pygments/formatter.py", line 92, in format return self.format_unencoded(tokensource, outfile) File "pygments/pygments/formatters/html.py", line 749, in format_unencoded for t, piece in source: File "pygments/pygments/formatters/html.py", line 618, in _wrap_div for tup in inner: File "pygments/pygments/formatters/html.py", line 631, in _wrap_pre for tup in inner: File "pygments/pygments/formatters/html.py", line 649, in _format_lines for ttype, value in tokensource: File "pygments/pygments/lexer.py", line 162, in streamer for i, t, v in self.get_tokens_unprocessed(text): File "pygments/pygments/lexer.py", line 503, in get_tokens_unprocessed for item in action(self, m): File "pygments/pygments/lexer.py", line 280, in callback match.group(i + 1)), ctx): File "pygments/pygments/lexer.py", line 328, in callback for i, t, v in lx.get_tokens_unprocessed(match.group(), **gt_kwargs): File "pygments/pygments/lexer.py", line 498, in get_tokens_unprocessed m = rexmatch(text, pos) TypeError: expected string or buffer


Comments (5)

  1. Alexander Pepper

    I ran into the same issue. Here my minimal example:

    $ echo "+ foo;" | pygmentize -l objectivec
    *** Error while highlighting:
    TypeError: expected string or buffer
       (file "/Library/Python/2.6/site-packages/Pygments-1.4dev_20111005-py2.6.egg/pygments/lexer.py", line 497, in get_tokens_unprocessed)

    It seems, that methods with NO stated arguments will fail.

    I also did some bisecting and nailed it to changes in cf606e81df8a and #638 .

  2. Log in to comment