Markdown filter: option not to extract code block content

[Chase Tingley]

See https://groups.yahoo.com/neo/groups/okapitools/conversations/topics/5347

In some content, content in code blocks is code that should not be translated. Since this behavior is not universal, the behavior should be optional.

[stevebpdx]

Hi Chase, I've been tasked by Lilt to fix this issue. Can you kindly provide a test file and some more context so that I can resolve this? Thanks, Steve

[Chase Tingley]

There is sample content in the yahoo group message.

[stevebpdx]

Hi Chase is it possible that you can provide an entire file that I can use to reproduce the problem? I need a little more context. What Okapi tool causes this issue or fails to handle it appropriately.

[Chase Tingley]

The issue is in the filter code, which can be accessed in a bunch of different ways. I often use tikal to test filters, although others use Rainbow.

A sample command to extract a markdown file to XLIFF using the markdown filter would be:

tikal.sh -fc okf_markdown -x test.md

[Chase Tingley]

Attached is a sample file based that is just the content from the mailing list.

If you run tikal on it as described above, you will see all the content extracted. This is incorrect, according to this bug report. It should be a filter option for the text in the code fencing to be extracted.

[Chase Tingley]

@stevebpdx Have you been able to make any progress on this? Let me know if I can help you out.

[stevebpdx]

Hi Chase, I actually could use some pointers on the Java IDE that is recommended for the Okapi Framework. Thanks!

[Chase Tingley]

Most people work in Eclipse.

To build a full distribution, you can look at the instructions in deployment/instructions.txt. What platform are you developing on? (I can give you more specific instructions.)

[stevebpdx]

I can develop in either Linux or Windows 10.

[Chase Tingley]

Ok. The artifacts build normally with Maven (mvn clean install from the top level), and you can use the Maven plugin in Eclipse to work with the projects. To run the integration tests and and build the applications, you'll need to do the full distribution build. You can do this by going to the deployment/maven and run either the update-and-rebuild-XXX.sh scripts or the rebuild-windows-XXX.bat script. These run the Maven scripts as well as some additional ant scripts to do other parts of the build that don't fit cleanly into the maven structure.

[Sun Hang]

@stevebpdx @tingley I uploaded more samples on https://spartansoftware.atlassian.net/servicedesk/customer/portal/5/OK-7

[Chase Tingley]

@superhell That's not a public site, but I can we can transfer your samples over here if it's ok with you.

[Sun Hang]

@tingley Sure. I uploaded samples to that site because I couldn't do it here.

[Chase Tingley]

@superhell Thanks, added. Can you quickly hit the 'Edit' button in the top right and see if you can see a "Select Attachments" section? It should be possible for users to add more attachments through that interface -- if you can't I will ask Yves about it.

[Sun Hang]

@tingley I can upload files now. Thanks.

[Chase Tingley]

I'm looking at this.

[Chase Tingley]

Fix issue #610 - Add new processing options to markdown filter - The translateCodeBlocks option (true by default) toggles extraction of fenced code blocks. - The translateHeaderMetadata option (false by default) toggles extraction of values from metadata headers.

This also upgrades flexmark to a newer version.

→ <<cset f9ad75e92c4a>>

[Chase Tingley]

Commit 20b6f00 makes this option also control whether inline code blocks are translated.