Some invalid symbols get ignored

Create issue
Issue #29 resolved
Fabian Maurer created an issue

The following code compiles without errors, and outputs 43. The invalid characters seems to be ignored.

testVar§$!=@42;
say "test", "$testVar";

Comments (9)

  1. Chimaine

    I can't seem to get ANTLR to actually recognize these characters to be there; they're just ignored. Has to be investigated further why this is a thing.

  2. mezzodrinker

    It appears that ANTLR does recognize the invalid characters correctly. For example, when running the following script in the Interactive Console…

    testVar§$!=€@42; print(testVar)
    

    …, the following errors are printed to the console…

    line 1:7 token recognition error at: '§'
    line 1:8 token recognition error at: '$'
    line 1:9 token recognition error at: '!'
    line 1:11 token recognition error at: '€'
    line 1:12 token recognition error at: '@'
    

    …and the script prints 42 to the output afterwards.

    Hence, it seems that the error recovery setting of the ANTLR lexer/parser only prints error to stderr without raising any errors in the Java code.

    I’m not sure if MTS sets the error recovery manually. Let’s dive into the code.

  3. mezzodrinker

    Issue is caused by the Mts3Lexer instance only having its default error listener (which prints any errors to stderr and continues as if nothing happened). By adding a custom error handler which functions similarly to the existing MtsAntlrErrorListener class, this issue can be prevented.

  4. Log in to comment