Markdown filter extraction/re-merge produces extra <ul> tag used in table cells.

[Kuro Kurosaka]

The attached file, ul-in-table-cell.md has a table cell that includes HTML unordered list element <ul>...</ul>. When tikal -x is applied to this file to generated the .md.xlf file, then tikal -m is applied to re-generated the .md file, the re-generated file has an extra <ul> tag.

• Original cell content: <ul><li>item1</li><li>item2</li></ul>
• Re-generated: <ul><ul><li>item1</li><li>item2</li></ul>

[Kuro Kurosaka]

Here is what I know so far. Markdown filter uses flexmark-java library to parse the MD document. It reads the MD document and build a tree of Nodes and returns it to Markdown filter. If it is a stand-alone run of HTML file occupying the entire line or lines, the entire thing is put together and one HtmlBlock node is built. The Markdown filter passes it to the HTML filter for further processing. But in the case of HTML tags within a table cell, this doesn't happen. Flexmark-java creates a node of type HtmlInline for each HTML tag, and currently the Markdown filter passes each node to HTML. So for this example, "<ul>" is passed to the HTML filter, "<li>", is passed to the HTML filter, and so forth. Because "<ul>" is a tag that requires the end tag, the HTML subfilter gets confused when it only see "<ul>". It tries to do something, and that something at this point is add another "<ul>" (not "</ul>"). The author of Flex-java confirmed that each HTML tag maps to an HtmlInline node is an intended behavior.

One possible solution is for Flexmark to detect a neighboring HtmlInline nodes and Text nodes, and putting together to create a longer span of HTML tags. To be exact, when a HtmlInline node is found, we start collecting the string, as long as the next node is either HtmlInline, HtmlComment, Text, until we see something else. Then we back up to the last HtmlInline node, and we pass the concatenated text to the HTML filter.

This most likely to have a side effect of mis-identifying the text nodes that is not meant to be part of a HTML block in theory, but it might worth trying.

[Kuro Kurosaka]

DirectShape.md has a <tbody> tag, <td> tags directly under <table> tag. They each appear twice after the xlf file is merged. Another report says if all empty lines are removed, duplication doesn't happen. It is suspected this may share the same cause.

[Kuro Kurosaka]

It turned out the issue with DirectShape.md is closely related to this issue but shows a more fundamental issue. A separate issue, issue 694, has been filed.

[Kuro Kurosaka]

It turned out the HTML filter has difficulty handling partial (unbalanced) HTML text only in its okf_html-wellformed variant, which Markdown filter's HTML subfilter configuration was based on. When I swtiched to an okf_html based configuration, insertion of an extra "ul" event stopped happening.