New EcmaScript lexer (close #1161)

#510 Open

Bitbucket cannot automatically merge this request.

The commits that make up this pull request have been removed.

Bitbucket cannot automatically merge this request due to conflicts.

Review the conflicts on the Overview tab. You can then either decline the request or merge it manually on your local system using the following commands:

hg update default
hg pull -r es6
hg merge es6
hg commit -m 'Merged in __item4__/pygments-main/es6 (pull request #510)'
  1. item4 item4

I rewrite JavaScript lexer for EcmaScript support.

This PR based on ECMA-262.

  • disable JavaScript lexer

    I do not remove JavaScript lexer because some lexers use it.

  • new EcmaScript lexer base

  • change mapping

  • add test case for yield syntax

  • add some real program based test cases

  • Issues #1161: new lexer request: ecmascript 6 new

Comments (4)

  1. Tim Hatch

    Report from Regexlint, which didn't find anything major:

    • Line 14: Zs includes 0020 already
    • Line 19: Newline needs to be escaped for this to work in Jython
    • Line 93: please escape right brace for this to work with the new Regex module (but see last point below)
    • Line 396: innerWidth included twice
    • Line 501: you want a rawstring; the parens are unnecessary
    • ES_GlobalConstructor especially, but any time you use '|'.join(foo); please use words() instead if it's literals (and you can remove backslashes at that point).

    I'll defer to @Georg Brandl on the best way to handle JavaScriptLexer compat; are there backward-compatibility issues that prevent us from just import EcmascriptLexer as JavascriptLexer here?