Markdown filter: Newlines lost in YAML metadata

[Christopher Cudennec]

Hi everyone!

When evaluating the new Markdown filter of 0.36-SNAPSHOT we came across a bug concerning YAML metadata: All newlines get lost after translation.

Example input:

---

title: Back to the future

year: 1985
---

Back to the Future is a 1985 American science fiction film directed by Robert Zemeckis and written by Zemeckis and Bob Gale.

Output of the filter's writer:

---title: Back to the futuretitle: year: 1985year: ---
Back to the Future is a 1985 American science fiction film directed by Robert Zemeckis and written by Zemeckis and Bob Gale.

You can take a look at the following commit to get a JUnit test for reproducing the problem: https://bitbucket.org/24t/okapi/commits/61956bbea6fa89e6ce410f0618da6a0542adfe1c?at=markdown-filter-yaml-header

[Kuro Kurosaka]

@ccudennec-24t , what platform did you run this test? On Mac, there is an issue of a blank line after the second line of three dashes disappears, but otherwise the newline was kept. Would it be possible that you run Okapi on Windows but the Markdown didn't have the Window's CR+LF end of line, or vice versa?

[Christopher Cudennec]

Hey Kuro!

• yes, I run the test on Windows.
• the test still fails if I change the EOL to CR + LF
• the problem only occurs within the YAML header - newlines within the rest of the document are not a problem

[Kuro Kurosaka]

Even on Mac, there is a problem. I used a version of the file from this report without any extra newlines, i.e.:

---
title: Back to the future
year: 1985
---
Back to the Future is a 1985 American science fiction film directed by Robert Zemeckis and written by Zemeckis and Bob Gale.

After extraction and merge without translation, the resulting file has the "Back to the Future is ..." sentence immediately after four hyphens without a newline.

---Back to the Future is

Probably adding addNewline(node) to the visitor of YamlFrontMatterBlock in MarkdownParser.java would fix this part of the bug.

[Beat Koch]

I have noticed the same problem as Christopher on macOS 10.13.6 with version M35. This is the beginning of my source file (which only has LF as line ending):

---
title: Hauptfenster
apple_title: Hauptfenster
description: Die Funktionen des Hauptfensters 
group: Referenz
order: 20
---

Das Fenster **Zahlungen** ist das Hauptfenster von PayMaker. Falls es geschlossen ist, können Sie es über das Menü **Zahlungen** → **Zahlungen anzeigen** öffnen. 

and this is what I get when I merge the translation:

---title: Fenêtre principaletitle: apple_title: Fenêtre principaleapple_title: description: Les fonctions de la fenêtre principaledescription: group: Référencegroup: order: 20order: ---La fenêtre **Paiements** est la fenêtre principale de PayMaker. Lorsqu'elle est fermée, vous pouvez l'ouvrir via le menu **Paiements** → **Afficher les paiements**.

[Deepak]

I have updated the YamlFrontMatterBlock visitor to add new lines. Please find the changes in commit 47e944d

[Kuro Kurosaka]

Commit ed7ace9 by pull request #245 fixes this with a minor issue of deleting blank lines, which will be tracked by a new issue #737.