Invalid indentation in lexer.get_tokens_unprocessed() in Pygments 1.6

Issue #956 invalid
tetburr
created an issue

The main 'else' statement within the for loop of get_tokens_unprocessed() is incorrectly indented, such that when m is None the else statement is not reached and the while loop runs again. This results in the following exception:

'Exception occurred: File "E:\Jython\jython2.5.2\Lib\site-packages\pygments-1.6-py2.5.egg\pygments\lexer.py", line 579, in get_tokens_unprocessed m = rexmatch(text, pos) RuntimeError: maximum recursion depth exceeded'

Resolution: - Indent the else block (lines 606 - 618) an extra 4 spaces.

Comments (3)

  1. Georg Brandl repo owner

    The indentation of the "else" clause is correct: it should only be executed after all rules of the current state have been tried.

    Of course you shouldn't hit the recursion limit during highlighting, but to debug that I'll need more information about the code you tried to highlight, and the lexer that was used. A full traceback would also be needed.

  2. Log in to comment