fortran lexer mis-parsing numeric literals

Issue #1472 new
David Carlisle
created an issue

The few digits of a float with a _ kind component are being interpreted as an integer so

1234.56b78_wp is producing <span class="mi">123</span><span class="mf">4.5678_wp</span>

also a float with no . but an e (or E) is being parsed as an error so

10E10 is producing <span class="mi">1</span><span class="err">0</span><span class="n">E10</span>

The following diff addresses both those (although other re-arrangements would be possible)

diff Pygments-2.2.0/pygments/lexers/fortran.py nag_fortran.py
158,160c158,161
<             (r'\d+(?![.e])(_[a-z]\w+)?', Number.Integer),
<             (r'[+-]?\d*\.\d+([ed][-+]?\d+)?(_[a-z]\w+)?', Number.Float),
<             (r'[+-]?\d+\.\d*([ed][-+]?\d+)?(_[a-z]\w+)?', Number.Float),
---
>             (r'\d+(?![.edED0-9])(_[a-z]\w+)?', Number.Integer),
>             (r'[+-]?\d*\.\d+([edED][-+]?\d+)?(_[a-z]\w+)?', Number.Float),
>             (r'[+-]?\d+\.\d*([edED][-+]?\d+)?(_[a-z]\w+)?', Number.Float),
>             (r'[+-]?\d+([edED][-+]?\d+)(_[a-z]\w+)?', Number.Float),

where the fist line is intended to stop an integer being immediately being followed by a digit, and the last is an extra clause for floats with no . (which then must have an e)

on a test

Program hmm

      Real (Kind=wp)  :: zzz


      zzz = 1234.5678_wp

      zzz = 1234.5678

      zzz = 10E10

End Program hmm

The existing lexer produces

bash-4.3$ pygmentize -f html -l fortran hmm.f90
<div class="highlight"><pre><span></span><span class="k">Program </span><span class="n">hmm</span>

      <span class="kt">Real</span> <span class="p">(</span><span class="nb">Kind</span><span class="o">=</span><span class="n">wp</span><span class="p">)</span>  <span class="kd">::</span> <span class="n">zzz</span>


      <span class="n">zzz</span> <span class="o">=</span> <span class="mi">123</span><span class="mf">4.5678_wp</span>

      <span class="n">zzz</span> <span class="o">=</span> <span class="mi">123</span><span class="mf">4.5678</span>

      <span class="n">zzz</span> <span class="o">=</span> <span class="mi">1</span><span class="err">0</span><span class="n">E10</span>

<span class="k">End Program </span><span class="n">hmm</span>
</pre></div>

the modified one produces

bash-4.3$ pygmentize -f html -l nag_fortran.py:FortranLexer -x hmm.f90
<div class="highlight"><pre><span></span><span class="k">Program </span><span class="n">hmm</span>

      <span class="kt">Real</span> <span class="p">(</span><span class="nb">Kind</span><span class="o">=</span><span class="n">wp</span><span class="p">)</span>  <span class="kd">::</span> <span class="n">zzz</span>


      <span class="n">zzz</span> <span class="o">=</span> <span class="mf">1234.5678_wp</span>

      <span class="n">zzz</span> <span class="o">=</span> <span class="mf">1234.5678</span>

      <span class="n">zzz</span> <span class="o">=</span> <span class="mf">10E10</span>

<span class="k">End Program </span><span class="n">hmm</span>
</pre></div>

Comments (0)

  1. Log in to comment