MatlabLexer returns Token.Error if function definition has ellipsis anywhere in it

Issue #952 new
Mark Mikofski
created an issue

the following are valid MATLAB/Octave/SciLab, but all fail:

function [ ...
    o1, o1] = my_fun(a1, a2)

function [o1, ...
    o1] = my_fun(a1, a2)

function [o1, o1] = ...
    my_fun(a1, a2)

function [o1, o1] = my_fun( ...
    a1, a2)

function [o1, o1] = my_fun(a1, ...
    a2)

add this regex (?:...\n[ \t]*)? where ever there is a chance of ellipsis? It would be nice if they were lexed as Token.Comments with whatever characters follow the ellipsis up to but not including the newline character

thanks!

Comments (8)

  1. Anonymous

    This isn't a fix for MatlabLexer, but if you want to typeset Matlab code nicely in LaTeX, consider using the matlab-prettifier package. It handles line continuation (...) without mishap, and it arguably does a better job at keeping track of context than MatlabLexer does. As a result, matlab-prettifier highlights code very similarly to the Matlab editor. See here for an example. I've also recently added a style that mimicks the output typically produced by MatlabLexer. Documentation for the package is on the way...

  2. Mark Mikofski reporter
    • AFAIK Pygments MATLABLexer appears to require the function to be contained in one line, and does not allow continuation lines.
    • Continuation lines are spec'd in Octave and MATLAB
    • I use the following regex to trap any continuation lines and replace them before passing them to the lexer:

      # functions must be contained in one line, no ellipsis, classdef is OK
      pat = r"""^[ \t]*(function)        # keyword
                ([\[\], \t\w.\n]*)       # outputs
                ([ \t=.\n]*)             # equal sign
                ([\w.]+)                 # name
                \(?([, \t\w.\n]*)\)?"""  # args
      pat = re.compile(pat, re.X | re.MULTILINE)
      repl = lambda m: m.group().replace('...\n', '')
      
    • Pygments MATLABLexer correctly parses MATLAB classdef, similar to Python's class keyword, regardless of continuation lines

    • RE: function spec - IMO the Octave function spec is more general, so all MATLAB function grammar would be a subset of Octave

    Hope that helps.

  3. Log in to comment