1. Georg Brandl
  2. pygments-main
  3. Issues
Issue #903 wontfix

PythonLexer adds new line character at the end of text

Michal Kandulski
created an issue

Hi,

When I parse python source without '\n' at the end with PythoLexer it adds additional (Token.Text, u'\n') at the end.

The following code snipet prints False :-(

from StringIO import StringIO
from pygments.lexers.agile import PythonLexer
from pygments.formatters.other import NullFormatter
out = StringIO()
codeIn = 'print "Hallo world!"'
NullFormatter().format(PythonLexer().get_tokens(codeIn), out)
codeOut = out.getvalue()
print codeIn == codeOut

If this is an issue its not nice, I think.

Michał

Comments (3)

  1. Gaurav Jain

    You need to disable the 'ensurenl' option which is set to True by default and adds a newline character at the end if not already present. Try:

    NullFormatter().format(PythonLexer(ensurenl=False).get_tokens(codeIn), out)
    

    Setting ensurenl=False resolved the issue for me

  2. Log in to comment