Does not flag trailing tab character; incompatibility with PyYAML.

I'm working on a project using SnakeYAML; I've tested this using v1.27, currently the latest.

We have a file that inadvertently ends with a tab character. Here is a reproducible test case:

test:
  - foo →
  - bar

Note that I used → to represent the tab character, Unicode U+0009, as in the YAML specification.

The Python PyYAML (I tested with 3.13) library will produce something like:

Found character '\t' that cannot start any token.

SnakeYAML however parses the above document with no complaint.

The YAML 1.1 specification in section 5.5. Miscellaneous Characters (referring to the s-ignored-space production) it says:

An ignored space character outside scalar content. Such spaces are used for indentation and separation between tokens. To maintain portability, tab characters must not be used in these cases, since different systems treat tabs differently. Note that most modern editors may be configured so that pressing the tab key results in the insertion of an appropriate number of spaces.

Indeed "Example 5.12. Invalid Use of Tabs" in the specification shows these example where tabs are and are not allowed. The explanatory text says (emphasis mine):

Tabs may appear inside comments and quoted or block scalar content. Tabs must not appear elsewhere, such as in indentation and separation spaces.

# Tabs do's and don'ts:
# comment: →
quoted: "Quoted →"
block: |
  void main() {
  →printf("Hello, world!\n");
  }
elsewhere:→# separation
→indentation, in→plain scalar

In particular the line beginning with "elsewhere" as explained by the text is not allowed.

I'm not an expert in the minutiae of the YAML 1.1 spec, but on the face of it this seems to say that PyYAML is correct and that SnakeYAML isn't correctly flagging an error for a tab character at the end of a line.

One might say that it's better to be lenient, but in the case of parsing interchangeable formats with clear specifications, leniency causes problems. In our case we're using multiple programming languages in our ecosystem. If we have a file that works fine with SnakeYAML but then later breaks somewhere down the data pipeline because of PyYAML, then this is a big problem for us. We need to be able to validate our YAML 1.1 files using SnakeYAML so that we don't get a surprise by PyYAML breaking with the same data.

Is my interpretation of the PyYAML 1.1 specification correct, and is the presence of a trailing tab on a YAML line truly a violation of the spec? If so, could SnakeYAML be fixed to correctly catch and report this error?

Thank you.

Comments (5)