Consider making the "python" lexer support python 3?

Issue #1415 new
Nathaniel Smith
created an issue

Python 3 is increasingly popular, and has new syntax that's used pretty frequently (like async/await, f-strings, ...). But many popular applications follow pygments's lead, and use the python lexer by default, which only supports python 2, so you get ugly rendering of code that uses these new constructs. (E.g.: pygmentize, sphinx, pypi's readme_renderer, ...)

Heck, AFAICT the python lexer doesn't even know about "{}".format(...) syntax, which is in python 2.

Generally the python3 lexer gives pretty decent results on python 2. Most of the things removed from python 3 are pretty rare (e.g. file), and some keywords turned into builtins (print, exec) and vice-versa (True, False, None), but most files look OK. Probably the biggest mis-rendering is that the python3 won't highlight unicode and xrange as builtins. Plus at this point I think it's clear that python 3 is where the long-term momentum is – e.g. django is now python 3 only. And the vast majority of projects are at the very least py2/py3 polyglots, and by definition the python3 lexer should work for such code. So I think there's a good case for renaming pythonpython2 and making python an alias for python3.

If that's considered too extreme, though, then maybe it would at least make sense to have a python lexer that takes the union of the two languages (so e.g. treating async and await as keywords, but also treating xrange as a builtin), and provide python2 and python3 lexers for those who want to be more specific.

Comments (1)

  1. Paul G

    Another option here might be to make it easy to globally set python to either python2 or python3 - for backwards compatibility you can keep the default to python2.

  2. Log in to comment