Issue #693 resolved

obj-c lexer fails on certain files

Aman Gupta avatarAman Gupta created an issue
>>> import pygments, pygments.lexers, pygments.formatters

>>> pygments.highlight("-initInManagedObjectContext:(NSManagedObjectContext*)aContext withUser:(User*)aUser\n", pygments.lexers.get_lexer_by_name("objective-c"), pygments.formatters.get_formatter_by_name("html"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pygments/pygments/", line 86, in highlight
    return format(lex(code, lexer), formatter, outfile)
  File "pygments/pygments/", line 66, in format
    formatter.format(tokens, realoutfile)
  File "pygments/pygments/", line 92, in format
    return self.format_unencoded(tokensource, outfile)
  File "pygments/pygments/formatters/", line 749, in format_unencoded
    for t, piece in source:
  File "pygments/pygments/formatters/", line 618, in _wrap_div
    for tup in inner:
  File "pygments/pygments/formatters/", line 631, in _wrap_pre
    for tup in inner:
  File "pygments/pygments/formatters/", line 649, in _format_lines
    for ttype, value in tokensource:
  File "pygments/pygments/", line 162, in streamer
    for i, t, v in self.get_tokens_unprocessed(text):
  File "pygments/pygments/", line 503, in get_tokens_unprocessed
    for item in action(self, m):
  File "pygments/pygments/", line 280, in callback + 1)), ctx):
  File "pygments/pygments/", line 328, in callback
    for i, t, v in lx.get_tokens_unprocessed(, **gt_kwargs):
  File "pygments/pygments/", line 498, in get_tokens_unprocessed
    m = rexmatch(text, pos)
TypeError: expected string or buffer

Comments (5)

  1. Alexander Pepper

    I ran into the same issue. Here my minimal example:

    $ echo "+ foo;" | pygmentize -l objectivec
    *** Error while highlighting:
    TypeError: expected string or buffer
       (file "/Library/Python/2.6/site-packages/Pygments-1.4dev_20111005-py2.6.egg/pygments/", line 497, in get_tokens_unprocessed)

    It seems, that methods with NO stated arguments will fail.

    I also did some bisecting and nailed it to changes in cf606e81df8a and #638 .

  2. Log in to comment
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.