[Literal style dump] Trailing Space prevents Literal style

[Anton Pryamostanov]

Dear Snakeyaml Team,

This is continuation of issue #436

Please find the below Java test case with only Snakeyaml dependency:

package apryamostanov.snakeyamltests;

import org.junit.Test;
import org.yaml.snakeyaml.DumperOptions;
import org.yaml.snakeyaml.Yaml;

public class LiteralStyleSpaceTest {

    @Test
    public void test() {
        DumperOptions options = new DumperOptions();
        options.setDefaultScalarStyle(DumperOptions.ScalarStyle.LITERAL);
        Yaml yaml = new Yaml(options);
        String correct = "this is some text with \"quotes\" abcd";
        String correctYaml = yaml.dump(correct);
        System.out.println(correctYaml);
        assert correctYaml.contains("|");
        String inCorrect = "this is some text with \"quotes\" abcd ";
        String inCorrectYaml = yaml.dump(inCorrect);
        System.out.println(inCorrectYaml);
        assert inCorrectYaml.contains("|");
    }

}

Actual result: second assertion fails Expected result: both assertions should pass

Note: the trailing space character prevents the string from Literal style dump. As per my understanding - it should not prevent.

[Andrey Somov]

Literal value is trimmed. Otherwise non-visible trailing spaces will become a part of the value.

[Andrey Somov]

Please be aware that chomping (|-) is supposed for new lines but not spaces

[Anton Pryamostanov]

Please share reference to YAML spec where it is saying that Literal value should be trimmed.

[Andrey Somov]

Sorry Anton. I was too quick. The issue is much more complex.

You can check that the trailing space is trimmed during parsing.

I think this is inderspecified in YAML 1.1 (but all the parsers do it)

YAML 1.2 is worse: it looks like the trailing spaces must stay. I strongly recommend you to raise your voice in the YAML mailing list to get a definitive answer (I do not remember why it is implemented like this. It was long time ago...)

[Anton Pryamostanov]

Hi Andrey,

Thank you for the feedback. I will clarify this in the YAML MGs and update here within few days.

[Anton Pryamostanov]

Hi Andrey,

I was about to ask question to YAML MG, but I realized that you are saying that white space is trimmed during parsing. Same is reflected in your test case.

In my test case there is no parsing.

new Yaml().dump Javadoc: Serialize a Java object into a YAML String.

So I think all 3 issues are valid and require fixing. So far there is no ambiguity when it comes to parsing interpretation - as it is not involved into the test case.

[Andrey Somov]

Dear Anton, if try to find mistakes in what I say instead of trying to understand then we do not go anywhere. You have already consumed quite an amount of our time and efforts.

If you show your irritation then it does not help you to solve anything. It is the opposite.

Please be aware that SnakeYAML is released more then 10 years ago and all the very basic stuff has already been resolved years ago. Even the bugs are already used by the community.

It does not help to create 10 issues when they are connected (and you know that they are connected!)

Back to the point.

Parsing and dumping work together. It must be symmetrical. If the parser ignores the trailing white space in literal it is useless to dump it.

[Andrey Somov]

Issue #438 was marked as a duplicate of this issue.

[Andrey Somov]

Issue #439 was marked as a duplicate of this issue.

[Anton Pryamostanov]

Hi Andrey,

Sorry if it seemed any irritation. It was unintentional from my side.

I will wait for the fix and let's hope it will cover all the questionable situations.

Many thanks. Appreciate your support and efforts on this awesome library.

[Andrey Somov]

What do you mean by waiting for the fix ? I do not see anything to fix.

[Anton Pryamostanov]

Sure, no problem. Please close this issue.

IMHO Output of Literal scalar style is one of the best features of YAML.

It is very unfortunate that you prefer not to support it for a major portion of scenarios (presence of several combinations of whitespace characters).

I will keep using your library in my project, and I will apply workarounds on my end (removing \r from the object fields before passing to Jackson/Snakeyaml).

As for the time, I did not consume any of your time. It is your social responsibility as an author of OSS library to take care of it. This is paid off with a reputation that you gain out of its popularity.

However I did consume my time, reporting the issues and performing debugging (hoping it will be useful for the other people using Snakeyaml and its maintainers), having used the Snakeyaml just for the first time in my project.

Cheers.

[Andrey Somov]

It is very unfortunate that you prefer not to support it for a major portion of scenarios (presence of several combinations of whitespace characters).

to support what ??? Why don't you want to raise the issue in the proper place to get the definitive answer ?

[Anton Pryamostanov]

I will give honest answer: I don't have enough knowledge to ask properly in YAML MG. I started to use YAML 1 week back for the 1st time.

I am referring to common sense and other implementations, such as Reference parser, where these scenarios work: http://ben-kiki.org/ypaste/data/1278/index.html http://ben-kiki.org/ypaste/data/1279/index.html

Both cases have spaces and CR LF. It parses OK.

I have given an idea of a potential major issue. I believe you can take it further and clarify if needed, and get more accurate feedback - as you know the people in the Mailing List.

[Andrey Somov]

I am referring to common sense

I do the same. Our common sense differ

nd other implementations, such as Reference parser, where these scenarios work:

I do not get it at all. What works ?

Both cases have spaces and CR LF. It parses OK.

parses or dumps ?

I have given an idea of a potential major issue.

Since you are the only one, may be there is no issue ? Otherwise it would be issue for everyone (everyone is using literal scalar)

[Anton Pryamostanov]

parses or dumps ?

http://ben-kiki.org/ypaste/ is official YAML reference parser. Thus it only parses. This is addressing your question if these scenarios should parse properly. If you still have doubts, you can consult Mailing Lists.

Since you are the only one, may be there is no issue ? Otherwise it would be issue for everyone (everyone is using literal scalar)

It is also really surprising for me that such issues exist. It seems other people didn't encounter '\r' in their data which was critical to be output as Literal. Not sure why. So I would have taken it as a luck that this is reported, even so late.

I do the same. Our common sense differ If it is problematic to fix this without breaking other stuff - I suggest to just document in javadoc of DumperOptions.ScalarStyle.LITERAL that several exceptions exist which at the moment can not be dumped as Literal.

This is a feasible alternative if the fix is breaking the other logic.

[Andrey Somov]

http://ben-kiki.org/ypaste/ is official YAML reference parser

It is YAML 1.2 parser !!! YAML 1.2 explicitly supports trailing white spaces in a literal scalar (which is not good, because it is reducing readability, but that is a separate issue)

[Andrey Somov]

It seems other people didn't encounter '\r' in their data which was critical to be output as Literal. Not sure why.

not only \r but any other white space (space, \t, \n) !

[Anton Pryamostanov]

It is YAML 1.2 parser !!! YAML 1.2 explicitly supports trailing white spaces in a literal scalar (which is not good, because it is reducing readability, but that is a separate issue)

As per YAML 1.2 spec:

YAML 1.2 is compatible with 1.1 for most practical applications - this is a minor revision

So if I understand correctly it's neither YAML 1.2 nor 1.1 parser. It is both.

[Anton Pryamostanov]

not only \r but any other white space (space, \t, \n) ! I have a question - didn't you encounter such cases during testing? Didn't it seem suspicious that Dumper does not use Literal for such cases? Because Literal that does not support \t does not make much sense.

[Andrey Somov]

So if I understand correctly it's neither YAML 1.2 nor 1.1 parser. It is both. you understand it incorrectly.

The fact that it is minor revision does not mean that your case stays the same.

I have a question - didn't you encounter such cases during testing? Didn't it seem suspicious that Dumper does not use Literal for such cases?I have a question - didn't you encounter such cases during testing? Didn't it seem suspicious that Dumper does not use Literal for such cases?

If you keep asking 100 times I keep answering 100 times. The current implementation seems to me correct (and apparently the community has the same opinion)